Kubernetes Backup

4 Feb 2025

The Kubernetes containerization platform processes and stores large volumes of data from various cluster components, including persistent storage blocks (Persistent Volumes), various manifests, and configuration files such as Deployments, ConfigMaps, and Secrets. It is important to organize backups to protect this data.

There are various solutions for simplifying the Kubernetes backup process. One of them is Velero, specifically designed to create Kubernetes cluster backups.

Today, we will take a detailed look at the process of creating backups using Velero.

Prerequisites

A deployed and running Kubernetes cluster. It can be a self-hosted cluster deployed or a Kubernetes cluster in the Hostman cloud.
Object storage for backup files. In this guide, we will use Hostman S3 object storage.
A server or a computer from which we will manage the cluster and install Velero. We'll use a machine with Ubuntu 24.04.
kubectl utility installed. The major version of kubectl should not differ from that of the cluster. For instance, if the cluster version is 1.31, you can use versions from 1.30 to 1.32. To download a specific version of kubectl, specify it in the URL, for example:

curl -LO https://dl.k8s.io/release/v1.32.0/bin/linux/amd64/kubectl

After installation, check the version:

kubectl version --client

Helm package manager installed. Helm simplifies installing, upgrading, and managing applications within a Kubernetes cluster. Helm organizes complex Kubernetes configurations into manageable packages called charts.

Creating S3 Storage

S3 is an object storage service for reliable storage of large datasets. Since Velero requires object storage, let's create one in the S3 Storage section of the Hostman management panel.

Click the Create button:

For this guide, we'll select the minimum storage size of 10 GB. In practice, you should choose a size that meets your needs. Set the storage type to Public. You can also rename the bucket if needed.

Velero Overview

Velero is an open-source client-server utility for creating backups and restoring Kubernetes cluster resources. It works with Kubernetes objects (such as Pods, Deployments, and Services) and saves them as snapshots. Additionally, it can back up data from Persistent Volume (PV) objects.

Velero Key Features:

Backup Creation: Save the state of the Kubernetes cluster, including manifests and Persistent Volumes.
Data Restoration: Restore the entire cluster or individual resources from a backup.
Data Migration: Move resources between Kubernetes clusters.

Velero Architecture

The Velero architecture consists of the following key components:

Velero Server (deployed inside the Kubernetes cluster): The server component runs as a Deployment object within the Kubernetes cluster. It handles backup and recovery tasks.
CLI (deployed outside the cluster): The client component provides a command-line interface for managing Velero and sends commands to the Velero server.
Cloud Storage Provider Plugins: Used to interact with data storage services (e.g., Amazon S3, Google Cloud Storage, and Azure Blob Storage).

Preparing the kubeconfig File

To connect to a cluster, you need the kubeconfig file — a special YAML file containing connection details for the cluster.

If you are using a Kubernetes cluster from Hostman, you can download the kubeconfig file from the Dashboard of your cluster.

Next, export the KUBECONFIG environment variable, specifying the full path to the kubeconfig file.

Linux and macOS

In the terminal, run the following command:

export KUBECONFIG=/root/Daring_Linnet_config.yaml

Windows

In the Windows PowerShell, use this command:

$env:KUBECONFIG = "C:\Users\alex\plugins\container-service\clusters\customername\Daring_Linnet_config.yaml"

Replace Daring_Linnet_config.yaml with the name of your kubeconfig file.

After exporting the environment variable, check the connection to the cluster by listing all available nodes:

kubectl get nodes

If the command returns a list of nodes, we have successfully connected to the cluster.

Installing Velero

Installing the Client Component

As mentioned earlier, Velero consists of a client (CLI) and a server component. We'll start by installing the client, which provides a command-line interface.

Download the .tar archive for the Velero client and extract it. We'll use version 1.15.1:

curl -L https://github.com/vmware-tanzu/velero/releases/download/v1.15.1/velero-v1.15.1-linux-amd64.tar.gz | tar -xz

The output will be a directory named velero-v1.15.1-linux-amd64 (where v1.15.1 is the version used). Move the directory to /usr/local/bin:

mv velero-v1.15.1-linux-amd64/velero /usr/local/bin/

Check the utility's functionality by displaying its version:

velero version

If the version is displayed, the client component has been successfully installed. Now we will proceed with the installation of the server component.

Installing the Server Component

One way to install the server component of Velero is through a Helm chart. To install Velero using Helm, follow these steps:

Create a new namespace named velero:

kubectl create namespace velero

Create a new Kubernetes Secret object to store the aws_access_key_id and aws_secret_access_key variables. These keys are essential for authenticating and authorizing access to S3 storage.

- S3 Access Key: A public identifier used to identify the user or application making the request.
- S3 Secret Access Key: A private key used to digitally sign requests. Keep this key confidential.

To find the S3 Access Key and S3 Secret Access Key, go to the S3 Storage section in the Hostman management panel and click on the bucket.

Copy these values and create a new file named velero-credentials-secret.yaml:

nano velero-credentials-secret.yaml

Add the following content:

apiVersion: v1
kind: Secret
metadata:
  name: cloud-credentials
  namespace: velero
type: Opaque
stringData:
  cloud: |
    [default]
    aws_access_key_id = UOY3beX5A3bV9Ly
    aws_secret_access_key = F3x78pH1d5BOu4BfVv

Create the secret in Kubernetes:

kubectl apply -f velero-credentials-secret.yaml

Add the official vmware-tanzu Helm repository:

helm repo add vmware-tanzu https://vmware-tanzu.github.io/helm-charts

Update the repository list:

helm repo update

List the repositories to confirm the addition:

helm repo ls

Install Velero using the following command:

helm install velero vmware-tanzu/velero \
  --namespace velero \
  --set credentials.existingSecret=cloud-credentials \
  --set 'configuration.backupStorageLocation[0].name=default' \
  --set 'configuration.backupStorageLocation[0].provider=aws' \
  --set 'configuration.backupStorageLocation[0].bucket=f60e2023-bucket-for-velero' \
  --set 'configuration.backupStorageLocation[0].config.region=us-2' \
  --set 'configuration.backupStorageLocation[0].config.s3ForcePathStyle=true' \
  --set 'configuration.backupStorageLocation[0].config.s3Url=https://s3.hmstorage.net' \
  --set 'configuration.volumeSnapshotLocation[0].name=default' \
  --set 'configuration.volumeSnapshotLocation[0].provider=aws' \
  --set 'configuration.volumeSnapshotLocation[0].config.region=us-2' \
  --set 'initContainers[0].name=velero-plugin-for-aws' \
  --set 'initContainers[0].image=velero/velero-plugin-for-aws:v1.7.0' \
  --set 'initContainers[0].volumeMounts[0].mountPath=/target' \
  --set 'initContainers[0].volumeMounts[0].name=plugins'

In the configuration.backupStorageLocation[0].bucket parameter, specify the bucket name, which you can find in the Hostman control panel.

Run the installation command. If there are no errors, a message will confirm that Velero has been deployed in the cluster. To monitor its status, use:

kubectl get deployment/velero -n velero

The deployment file is successfully launched, as indicated by the READY and UP-TO-DATE statuses.

You can also check the status of the Velero pod:

kubectl get pods -n velero

If the pod is running, you can optionally check its logs (where velero-7bb8d5c5f-jwg5c is the Velero pod name):

kubectl logs velero-7bb8d5c5f-jwg5c -n velero

The Velero installation is now fully complete.

Backup Using Velero

To test the backup process, we will create a new namespace and several Kubernetes objects within it.

Create a namespace named test-velero:

kubectl create ns test-velero

Create a Deployment file with two containers running the NGINX web server and a LoadBalancer service.

nano nginx-dev.yaml

Add the following configuration:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-dev
  namespace: test-velero
  labels:
    app: nginx
spec:
  replicas: 2
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - image: nginx:1.17.6
        name: nginx
        ports:
        - containerPort: 80
---
apiVersion: v1
kind: Service
metadata:
  labels:
    app: nginx
  name: nginx-test-service
  namespace: test-velero
spec:
  ports:
  - port: 80
    targetPort: 80
  selector:
    app: nginx
  type: LoadBalancer

Apply the file and create the resources:

kubectl apply -f nginx-dev.yaml

Verify the status of the created resources:

kubectl get all -n test-velero

Creating a Backup

To create a backup for all resources in the test-velero namespace, run the following command:

velero backup create nginx-test-backup --include-namespaces test-velero

If the backup was created successfully, you will see the following message:

Backup request "nginx-test-backup" submitted successfully.
Run `velero backup describe nginx-test-backup` or `velero backup logs nginx-test-backup` for more details.

You can check the status with the describe command:

velero backup describe nginx-test-backup

If successful, the status will be Completed.

Listing Backups

To view all backups in the storage, run:

velero backup get

The output will display the status (STATUS), number of errors (ERRORS), warnings (WARNINGS), creation time (CREATED), and expiration time (EXPIRES) for each backup.

Restoring a Backup

To test the restoration process, first delete the previously created namespace and all objects within it:

kubectl delete namespace test-velero

Restore the backup by specifying its name (nginx-test-backup):

velero restore create --from-backup nginx-test-backup

Check the restoration status using the following command, providing the name of the restored copy (obtained from the velero restore create output):

velero restore describe nginx-test-backup-20250114155656

If successful, the status will be Completed.

Viewing Backup Files

To view backup files, navigate to the Objects tab in the S3 Storage section in your Hostman control panel.

Velero creates separate directories for:

Backups: containing backup data for the respective resources.
Restorations: containing details about restored objects.

Each directory contains the corresponding Kubernetes objects for backup and restoration purposes.

Useful Commands for Backup with Velero

Velero offers extensive backup functionality, allowing you to create backups for specific objects or configurations. Below are some useful examples:

Scheduled Backup for Specific Namespaces

To automatically create backups for all objects in the default and my-namespace namespaces every day at 2:00 AM:

velero schedule create daily-backup --schedule="0 2 * * *" --include-namespaces default,my-namespace

Backup for Specific Resources

To create a backup only for objects of type deployment in the default namespace:

velero backup create my-backup2 --include-resources deployments --include-namespaces default

Full Cluster Backup

To back up the entire Kubernetes cluster, including cluster-scoped resources such as ClusterRole, ClusterRoleBinding, CustomResourceDefinition (CRD), PersistentVolume, and StorageClass:

velero backup create full-cluster-backup

Backup by Label Selector

To back up only objects with a specific label, for instance, those with the selector app=nginx:

velero backup create backup-with-label-nginx --selector "app=nginx"

Backup Excluding a Label Selector

To back up only objects without a specific label selector, such as excluding objects labeled app=nginx:

velero backup create backup-with-no-label-nginx --selector "app=nginx"

Excluding a Specific Namespace

To exclude the kube-system namespace and all its objects from the backup:

velero backup create backup-exclude-kube-system --exclude-namespaces kube-system

Excluding Specific Resources

To exclude all secrets from the backup:

velero backup create backup-exclude-secrets --exclude-resources secrets

Before running production backups, validate node, pod, and volume health as described in Kubernetes Cluster Health Checks—covering viewing detailed information about resources and various components to ensure all resources are ready.

Conclusion

In this practical guide, we covered how to install Velero and how to use it to create Kubernetes backups and restore data. Velero's rich functionality allows for quick and straightforward backup-related tasks, making it a valuable tool for maintaining data safety and cluster reliability.

Kubernetes Backup

Prerequisites Copy link

Creating S3 Storage Copy link

Velero Overview Copy link

Velero Architecture Copy link

Preparing the kubeconfig File Copy link