Integrating Prometheus with Grafana Over Kubernetes.

6 min readSep 10, 2020

In this project, we will integrate Prometheus which is a monitoring tool with Grafana which is Visualizing tool over Kubernetes which is a Container Orchestration tool. Before starting over it, let's see briefly what are these tools about & whats the purpose of their integration.

What is Prometheus?

Prometheus is a free software application used for event monitoring and alerting. It records real-time metrics in a time series database built using an HTTP pull model, with flexible queries and real-time alerting.

Prometheus collects metrics from monitored targets by scraping metrics from HTTP endpoints on these targets. Prometheus collects data in the form of time series, hence also known as TSDB i.e., Time Series DataBase. The time series are built through a pull model: the Prometheus server queries a list of data sources at a specific polling frequency.

What is Grafana?

Grafana is open-source visualization and analytics software. It allows you to query, visualize, alert on, and explore your metrics no matter where they are stored. In plain English, it provides you with tools to turn your time-series database (TSDB) data into beautiful graphs and visualizations. Hence we can also provide the Data fetched by Prometheus from any target to create visuals in Grafana. Grafana can almost create visuals from the DB provided all the Data Source like Prometheus, Influx DB, ElasticSearch, MySQL, Graphite, OpenTSDB, etc.

What is Kubernetes?

Kubernetes is a portable, extensible, open-source platform for managing containerized workloads and services, that facilitates both declarative configuration and automation. It has a large, rapidly growing ecosystem. Kubernetes services, support, and tools are widely available. It is a Container Orchestration tool, which helps launch application over Containers which gives more power and flexibility & helps us manage them very easily.

About the Project:

@ Task 5: Integrate Prometheus and Grafana and perform in the following way:

Deploy them as pods on top of Kubernetes by creating resources Deployment, ReplicaSet, Pods or Services.
And make their data to remain persistent.
And both of them should be exposed to the outside world.

Some Pre-requisites for this project:

Here we are going to use K8S service from Minikube, so download & installed Minikube.
Concepts of Prometheus, Graphana & K8S will be used directly.

◉ I have Performed the Task and attached the screenshots for your reference:

Step 1: Start the Minikube VM so that we can use Kubernetes (K8S) inside it.

Step 2: Config file for Minikube, which contains the server of K8S cluster.

Step 3: Let’s write YAML files for Deployment, PVC, Service in K8S for Prometheus & Grafana.

# Firstly creating YAML file for Prometheus application including PVC, Service & Deployment.

— firstly creating Service with NodePort type exposed to port no. 30080 & the port no. for Prometheus application is 9090. Then will create a PVC to make data persistent & finally the Pods containing Prometheus app containing the actual app.

apiVersion: v1
kind: Service
metadata:
  name: prom-svc
  labels:
    app: prom
spec:
  selector:
    app: prom
  type: NodePort
  ports:
    - nodePort: 30080
      port: 9090
      targetPort: 9090
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: prom-pvc
  labels:
    name: prompvc
spec:
   accessModes:
    - ReadWriteOnce
   resources:
     requests:
        storage: 5Gi
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: prom-deploy
  labels:
    app: prom
spec:
  replicas: 1
  selector:
    matchLabels:
      job: monitor
      app: prom
  strategy:
    type: Recreate
  template:
    metadata:
      labels:
        job: monitor
        app: prom
    spec:
      containers:
      - image: vimal13/prometheus
        name: prom-con
        ports:
        - containerPort: 9090
          name: prom-con
        volumeMounts:
        - name: prom-persistent-storage
          mountPath: /etc/prometheus/data
      volumes:
      - name: prom-persistent-storage
        persistentVolumeClaim:
          claimName: prom-pvc

# Now create YAML file for Grafana application including PVC, Service & Deployment.

— creating Service with NodePort type exposed to port no. 30050 & the port no. for Grafana application is 3000. Then will create a PVC to make data persistent & finally the Pods containing Grafana app containing the actual app.

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: grafana-pvc
  labels:
    name: grafanapvc
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 2Gi
---
apiVersion: v1
kind: Service
metadata:
  labels:
    app: grafana
  name: grafana-svc
spec:
  type: NodePort
  selector:
    app: grafana
  ports:
    - nodePort: 30050
      port: 3000
      targetPort: 3000
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: grafana-deploy
  labels:
    app: grafana
spec:
  replicas: 1
  selector:
    matchLabels:
      app: grafana
      job: visuals
  strategy:
    type: Recreate
  template:
    metadata:
      labels:
        app: grafana
        job: visuals
    spec:
      containers:
      - image: vimal13/grafana
        name: graf-cont
        ports:
        - containerPort: 3000
          name: graf-cont
        volumeMounts:
        - name: grafana-persistent-storage
          mountPath: /var/lib/grafana
      volumes:
      - name: grafana-persistent-storage
        persistentVolumeClaim:
          claimName: grafana-pvc

# Create another YAML file known as Kustomization:

— this will run all the other YAML file which we list in this file, just by running this file itself instead of running each file one by one.

apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
  - prom.yml
  - grafana.yml

Step 3: Now run the Kustomization file:

— this will launch K8S resources like deployment, PVC & Service for Prometheus & Grafana application. To run the Kustomisation.yaml file run this cmd:

> kubectl apply -k .

# All the Pods, Deployment, VPC & Service launched, as we can see here.

# Check the Minikube IP to access the Prometheus & Grafana app:

Step 4: Let’s access our Prometheus & Grafana app and create some visuals:

# Accessing Prometheus WebUI, which we exposed over port no. 30080

# Accessing Grafana WebUI, which we exposed over port no. 30050

Step 5: Writing Prom Queries & creating Visuals in Grafana:

# Creating Dashboard in Grafana with Data source as Prometheus

# Enter the URL of Prometheus Data Source so that Grafana can access it for the info accumulated by Prometheus to create Visuals.

# Written various Queries to create Visuals through them in Grafana Dashboard.

◉ Conclusion:

This project helped us learn to integrate Prometheus which is monitoring tool & Grafana which is a visualization tool over Kubernetes (K8S). Kubernetes is a great platform which helps us launch & destroy OS in seconds over containers & manage it very easily, which is very rapidly used technologies in today world. Visuals are very important to help us easily track on things or to monitor something. I would thank Vimal Sir for providing me with such a great project which helped me learn a lot.