VictoriaMetrics Operator

VictoriaMetrics Operator is a Kubernetes operator that automates the deployment and management of monitoring components based on VictoriaMetrics. It allows you to create and configure individual services for collecting, storing, and visualizing metrics.

The operator uses CRDs (Custom Resource Definitions) to manage monitoring infrastructure. The main components include:

• VMSingle — a single-instance VictoriaMetrics component responsible for storing metrics. It’s suitable for simple projects and low-traffic clusters.
• VMCluster — a scalable setup with storage replicas and proxy components. Best suited for production and high-volume environments.
• VMAgent — collects metrics from pods, services, and other sources, then sends them to VMSingle or VMCluster via remoteWrite.
• VMPodScrape — a CRD object that defines which pods should be scraped for metrics, and on what addresses/ports.

Full documentation for VictoriaMetrics Operator is available on the official website.

Installation

To install VictoriaMetrics Operator:

1. Go to the Kubernetes section and select your cluster.
2. On the Addons tab, click VictoriaMetrics Operator.
3. In the configuration window, you can optionally adjust parameters, such as setting a custom namespace, changing component versions, or disabling unnecessary CRDs.
4. After configuration, click Install.

Once installed, the cluster will include the victoria-metrics-operator namespace and all necessary CRDs, including VMSingle, VMAgent, VMPodScrape, and others. To list them, run:

kubectl get crd | grep 'vm'

Usage Example

In this example, we will:

1. Deploy a VMSingle instance to store metrics.
2. Set up a VMAgent to collect metrics from sources.
3. Launch a pod with Nginx and nginx-prometheus-exporter to expose metrics.
4. Create a VMPodScrape object to allow VMAgent to collect those metrics.
5. Configure Ingress to access the VictoriaMetrics web UI.

You’ll need the Nginx Ingress addon installed in your cluster for this example.

Step 1: Create VMSingle

This component will accept and store metrics. In the example, it’s configured with minimal resource requirements and stores data in memory for 1 day.

Create a file called vmsingle.yaml with the following configuration.

apiVersion: operator.victoriametrics.com/v1beta1
kind: VMSingle
metadata:
  name: vm-single
  namespace: victoria-metrics-operator
spec:
  retentionPeriod: "1"
  resources:
    requests:
      cpu: 100m
      memory: 256Mi

Step 2: Create VMAgent

This agent will collect metrics and forward them to VMSingle. Create a file named vmagent.yaml:

apiVersion: operator.victoriametrics.com/v1beta1
kind: VMAgent
metadata:
  name: vmagent
  namespace: victoria-metrics-operator
spec:
  remoteWrite:
    - url: http://vmsingle-vm-single.victoria-metrics-operator:8428/api/v1/write
  scrapeInterval: 30s
  selectAllByDefault: true
  port: "8429"

Step 3: Deploy Nginx and Exporter

Launch a pod with Nginx and nginx-prometheus-exporter, which will expose metrics via /stub_status.

Create a file called nginx.yaml:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx
  namespace: victoria-metrics-operator
spec:
  replicas: 1
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
        - name: nginx
          image: nginx:1.25
          ports:
            - containerPort: 80
              name: http
          volumeMounts:
            - name: nginx-conf
              mountPath: /etc/nginx/conf.d/default.conf
              subPath: default.conf
        - name: exporter
          image: nginx/nginx-prometheus-exporter:1.1.0
          args:
            - "-nginx.scrape-uri=http://localhost/stub_status"
          ports:
            - containerPort: 9113
              name: metrics
      volumes:
        - name: nginx-conf
          configMap:
            name: nginx-conf
---
apiVersion: v1
kind: ConfigMap
metadata:
  name: nginx-conf
  namespace: victoria-metrics-operator
data:
  default.conf: | server { listen 80; location / { return 200 'OK'; } location /stub_status { stub_status; allow 127.0.0.1; deny all; } } ---
apiVersion: v1
kind: Service
metadata:
  name: nginx
  namespace: victoria-metrics-operator
spec:
  selector:
    app: nginx
  ports:
    - name: http
      port: 80
      targetPort: http
    - name: metrics
      port: 9113
      targetPort: metrics

Step 4: Configure VMPodScrape

This object tells VMAgent which pods to scrape metrics from. It will target pods with the label app: nginx and query the metrics port at /metrics.

Create a file named vmpodscrape.yaml:

apiVersion: operator.victoriametrics.com/v1beta1
kind: VMPodScrape
metadata:
  name: nginx-scrape
  namespace: victoria-metrics-operator
spec:
  selector:
    matchLabels:
      app: nginx
  podMetricsEndpoints:
    - port: metrics
      path: /metrics
      interval: 30s

Step 5: Set Up Ingress for UI

Before configuring Ingress, ensure your cluster has a service of type LoadBalancer for external access. You can check with:

kubectl get svc -n ingress-nginx

If none is present, create it:

apiVersion: v1
kind: Service
metadata:
  name: nginx-loadbalancer
  namespace: ingress-nginx
spec:
  selector:
    app.kubernetes.io/name: ingress-nginx
  ports:
    - port: 80
      targetPort: 80
      appProtocol: k8s.hostman.com/proto-http
  type: LoadBalancer

After creation, retrieve the external IP:

kubectl get svc -n ingress-nginx nginx-loadbalancer

Use that IP as an A record for the domain where you want the VictoriaMetrics UI to be accessible.

Step 6: Create Ingress for VictoriaMetrics UI

Specify the domain in the host field of your Ingress manifest. Create a file called vm-ui-ingress.yaml:

apiVersion: v1
kind: Service
metadata:
  name: nginx-loadbalancer
  namespace: ingress-nginx
spec:
  selector:
    app.kubernetes.io/name: ingress-nginx
  ports:
    - port: 80
      targetPort: 80
      appProtocol: k8s.hostman.com/proto-http
  type: LoadBalancer

Apply all the created manifests:

kubectl apply -f vmsingle.yaml
kubectl apply -f vmagent.yaml
kubectl apply -f nginx.yaml
kubectl apply -f vmpodscrape.yaml
kubectl apply -f vm-ui-ingress.yaml

Accessing the UI

Navigate to the domain specified in your Ingress (e.g., http://vm.example.com). 

Once there:

1. Select vmui to open the VictoriaMetrics web interface.
2. In the query field, type a metric name, such as up, and click Execute Query.
3. If everything is configured correctly, you’ll see metrics being collected from the Nginx pod.