Kubernetes Cluster Health Checks
Kubernetes
28.01.2025
Reading time: 11 min
Share
The Kubernetes containerization platform is a complex system consisting of many different components and internal API objects totaling over 50. When issues arise with the cluster, it is important to know how to troubleshoot them. There are many different health checks available for a Kubernetes cluster and its components — let's go over them today.
Connecting to a Kubernetes Cluster with kubectl
To connect to a Kubernetes cluster using the kubectl command-line utility, you need a kubeconfig configuration file that contains the settings for connecting to the cluster. By default, this file is located in the hidden .kube directory in the user's home directory. The configuration file is located on the master node at /etc/kubernetes/admin.conf.
To copy the configuration file to the user's home directory, you need to run the following command:
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/configWhen using cloud-based Kubernetes clusters, you can download the file from the cluster's management panel. For example, on Hostman:
After copying the file to the user's home directory, you need to export the environment variable so that the kubectl utility can locate the configuration file. To do this, run:
export KUBECONFIG=$HOME/.kube/configNow, the kubectl command will automatically connect to the cluster, and all commands will be applied to the cluster specified in the exported configuration file.
If you're using a kubeconfig file downloaded from the cluster's management panel, you can use the following command:
export KUBECONFIG=/home/user/Downloads/config.yamlWhere /home/user/Downloads/ is the full path to the config.yaml file.
Checking the Client and Server Versions of Kubernetes
Although this check might seem not so obvious, it plays a fundamental role in starting the troubleshooting process. The reason is that for Kubernetes to function stably, the client and server versions of Kubernetes need to be identical to avoid unexpected issues. This is mentioned in the official kubectl installation documentation.
To check the client and server version of your Kubernetes cluster, run the following command:
kubectl version
In the output of the command, pay attention to the Client Version and Server Version lines. If the client and server versions differ (as in the screenshot above), the following warning will appear in the command output:
WARNING: version difference between client (1.31) and server (1.29) exceeds the supported minor version skew of +/-1.Retrieving Basic Cluster Information
During cluster health checks, it might be useful to know the IP address or domain name of the control plane component, as well as the address of the embedded Kubernetes DNS server — CoreDNS. To do this, use the following command:
kubectl cluster-info
For the most detailed information about the cluster, you can obtain a cluster dump using the command:
kubectl cluster-info dump
Note that this command produces a huge amount of data. For further use and analysis, it's a good idea to save the data to a separate file. To do this, redirect the output to a file:
kubectl cluster-info dump > cluster_dump.txtRetrieving All Available Cluster API Objects
If you need to get a list of all the API objects available in the cluster, run the following command:
kubectl api-resources
Once you know the names of the cluster objects, you can perform various actions on them — from listing the existing objects to editing and deleting them. Instead of using the full object name, you can use its abbreviation (listed in the SHORTNAMES column), though abbreviations are not supported for all objects.
Cluster Health Check
Let's go over how to check the health of various components within a Kubernetes cluster.
Nodes Health Check
Checking Node Status
Start by checking the status of the cluster nodes. To do this, use the following command:
kubectl get nodesIn the output, pay attention to the STATUS column. Each node should display a Ready status.
Viewing Detailed Information
If any node shows a NotReady status, you can view more detailed information about that node to understand the cause. To do this, use the command:
kubectl describe node <node_name>In particular, pay attention to the Conditions and Events sections, which show all events on the node. These messages can help determine the cause of the node's unavailability.
Additionally, the Conditions section displays the status of the following node components:
-
NetworkUnavailable— Shows the status of the network configuration. If there are no network issues, the status will be False. If there are network issues, it will be True. -
MemoryPressure— Displays the status of memory usage on the node. If sufficient memory is available, the status will be False; if memory is running low, the status will be True. -
DiskPressure— Displays the status of available disk space on the node. If enough space is available, the status will be False. If disk space is low, the status will be True. -
PIDPressure— Shows the status of process "overload." If there are only a few processes running, the status will be False. If there are many processes running, the status will be True. -
Ready— Displays the overall health of the node. If the node is healthy and ready to run pods, the status will be True. If any issues are found (e.g., memory or network problems), the status will be False.
Monitoring Resource Usage
In Kubernetes, you can track resource consumption for both cluster nodes and pod-type objects, as well as containers.
To display resource usage consumed by the cluster nodes, use the command:
kubectl top node
The top node command shows how much CPU and memory each node is consuming. The values are displayed in millicores for CPU and bytes for memory, and also as percentages.
To display resource usage of all pods across all namespaces running in the cluster, use the command:
kubectl top pod -A
If you need to display resource usage for pods in a specific namespace, specify the namespace with the -n flag:
kubectl top pod -n kube-system
To view the resource consumption specifically by containers running in pods, use the --containers option:
kubectl top pod --containers -A
Viewing Events in the Cluster
To view all events within the cluster, use the following command:
kubectl get events
It will display all events, regardless of the type of object in the cluster. The following columns are used:
LAST SEEN— The time when the event occurred, displayed in seconds, minutes, hours, days, or months.TYPE— Indicates the event's status, which is akin to the severity level. The supported statuses are:Normal,Warning, andError.REASON— Represents the cause of the event. For example,Startingindicates that an object in the cluster was started, andPullingmeans that an image for a container was pulled.OBJECT— The cluster object that triggered the event, including nodes in the cluster (e.g., during initialization).MESSAGE— Displays the detailed message of the event, which can be useful for troubleshooting.
To narrow down the list of events, you can use a specific namespace:
kubectl get events -n kube-system
For more detailed event output, use the wide option:
kubectl get events -o wide
The wide option adds additional columns of information, including:
SUBOBJECT— Displays the subobject related to the event (e.g., container, volume, secret).SOURCE— The source of the event, which could be components likekubelet,node-controller, etc.FIRST SEEN— The timestamp when the event was first recorded in the cluster.COUNT— The number of times the event has been repeated since it was first seen in the cluster.NAME— The name of the object (e.g., pod, secret) associated with the event.
To view events in real-time, use the -w flag:
kubectl get events -w
The get events command also supports filtering via the --field-selector option, where you specify a field from the get events output. For example, to display all events with a Warning type in the cluster:
kubectl get events --field-selector type=Warning -A
Additionally, filtering by timestamps is supported. To display events in the order they first occurred, use the .metadata.creationTimestamp parameter:
kubectl get events --sort-by='.metadata.creationTimestamp'Monitoring Kubernetes API Server
The API server is a critical component and the "brain" of Kubernetes, that processes all requests to the cluster. The API server should always be available to respond to requests. To check its status, you can use special API endpoints: livez and readyz.
To check the live status of the API server, use the following command:
kubectl get --raw '/livez?verbose'
To check the readiness status of the API server, use the following command:
kubectl get --raw '/readyz?verbose'
If both the livez and readyz requests return an ok status, it means the API server is running and ready to handle requests.
Kubernetes Cluster Components
To quickly display the status of all cluster components, use the following command:
kubectl get componentstatuses
If the STATUS and MESSAGE columns show Healthy and ok, it means the components are running successfully. If any component encounters an error or failure, the STATUS column will display Unhealthy, and the MESSAGE column will provide an error message.
Container Runtime
As is known, Kubernetes itself does not run pods with containers. Instead, it uses an external component called the Container Runtime Interface (CRI), or simply the container runtime. It’s important to ensure that the container runtime environment is functioning correctly. At the time of writing, Kubernetes supports the following container runtimes:
- containerd
- CRI-O
It’s worth mentioning that Docker's container runtime is no longer supported starting from Kubernetes version 1.24.
CRI-O
First, check the status of the container runtime. To do this, on the node where the error appears, run the following command:
systemctl status crio
In the Active line, the status should show active (running). If it shows failed, further investigation is needed in the crio information messages and log files. To display the basic information about crio, including the latest error messages, use the command:
crictl info
If an error occurs while using crio, the message parameter will show a detailed description. crio also logs all its activities to log files, typically found in the /var/log/crio/pods directory.
Additionally, you can use the journalctl logs. To display all logs for the crio unit, run:
journalctl -u crioContainerd
As with crio, start by checking the status of the container runtime. On the node where the error appears, run:
systemctl status containerd
In the Active line, the status should show active (running). If the status shows failed, you can get more detailed information by using the built-in status command, which will display all events, including errors:
containerd statusAlternatively, you can view the logs using journalctl. To display logs for the containerd unit, run:
journalctl -u containerdYou can also check the configuration file parameters for containerd using two commands (the output is usually quite large):
-
containerd config default— Displays the default configuration file. Use this if no changes have been made to the file. If errors occur, this file can be used for rollback. -
containerd config dump— Displays the current configuration file, which may have been modified.
Pods Health Check
Kubernetes operates with pods, the smallest software units in the cluster, where containers with applications are run. The status of pods should always be READY. To display a list of all pods in the cluster and their statuses, use the following command:
kubectl get po -ATo display pods in a specific namespace, use the -n flag followed by the namespace name:
kubectl get po -n kube-systemFor more detailed information about a pod, including any possible errors, use the kubectl describe pod command, which provides the most detailed information about the pod:
kubectl describe pod coredns-6997b8f8bd-b5dq6 -n kube-systemAll events related to the pod, including errors, are displayed in the Events section.
Getting Information About Objects with kubectl describe
The kubectl describe command is a powerful tool for finding detailed information about an object, including searching for and viewing various errors. You can apply this command to all Kubernetes objects that are listed in the output of the kubectl api-resources command.
Deployment files are widely used when deploying applications in a Kubernetes cluster. They allow you to control the state of service deployments, including scaling application replicas. To display the statuses of all available deployments in the Kubernetes cluster, use the command:
kubectl get deployments -A
It is important that the columns READY, UP-TO-DATE, and AVAILABLE display the same number of pods as specified in the deployment file. If the READY column shows 0 or fewer pods than specified, the pod with the application will not be started. To find the error's cause, use the describe command with the type of object, in this case, deployment:
kubectl describe deployment coredns -n kube-systemJust like when using describe for pods, all events, including errors, are displayed in the Conditions section.
Conclusion
Checking the health of a Kubernetes cluster is an important step in troubleshooting and resolving issues. Kubernetes consists of many different components, each with its own verification algorithm. It is important to know what and how to check to identify and fix errors quickly.
Kubernetes
28.01.2025
Reading time: 11 min
Share
Similar
Kubernetes
Kubernetes Requests and Limits
When working with the Kubernetes containerization platform, it is important to control resource usage for cluster objects such as pods. The requests and limits parameters allow you to configure resource consumption limits, such as how many resources a pod can use in a Kubernetes cluster. This article will explore the use of requests and limits in Kubernetes through practical examples.
Prerequisites
To work with requests and limits in a Kubernetes cluster, we need:
A Kubernetes cluster (you can create one in the Hostman control panel). For testing purposes, a cluster with two nodes will suffice. The cluster can also be deployed manually by renting the necessary number of cloud or dedicated (physical) servers, setting up the operating system, and installing the required packages.
Lens or kubectl for connecting to and managing your Kubernetes clusters.
Connecting to a Kubernetes Cluster Using Lens
First, go to the cluster management page in your Hostman panel. Download the Kubernetes cluster configuration file (the kubeconfig file).
Once Lens is installed on your system, launch the program, and from the left menu, go to the Catalog (app) section:
Select Clusters and click the blue plus button at the bottom right.
Choose the directory where you downloaded the Kubernetes configuration file by clicking the Sync button at the bottom right.
After this, our cluster will appear in the list of available clusters. Click on the cluster's name to open its dashboard:
What are Requests and Limits in Kubernetes
First, let's understand what requests and limits are in Kubernetes.
Requests are a mechanism in Kubernetes that is responsible for allocating physical resources, such as memory and CPU cores, to the container being launched. In simple terms, requests in Kubernetes are the minimum system requirements for an application to function properly.
Limits are a mechanism in Kubernetes that limits the physical resources (memory and CPU cores) allocated to the container being launched. In other words, limits in Kubernetes are the maximum values for physical resources, ensuring that the launched application cannot consume more resources than specified in the limits. The container can only use resources up to the limit specified in the Limits.
The request and limit mechanisms apply only to objects of type pod and are defined in the pod configuration files, including deployment, StatefulSet, and ReplicaSet files.
Requests are added in the containers block using the resources parameter. In the resources section, you need to add the requests block, which consists of two values: cpu (CPU resource request) and memory (memory resource request). The syntax for requests is as follows:
containers:
...
resources:
requests:
cpu: "1.0"
memory: "150Mi"
In this example, for the container to be launched on a selected node in the cluster, at least one free CPU core and 150 megabytes of memory must be available.
Limits are set in the same way. For example:
containers:
...
resources:
limits:
cpu: "2.0"
memory: "500Mi"
In this example, the container cannot use more than two CPU cores and no more than 500 megabytes of memory.
The units of measurement for requests and limits are as follows:
CPU — in millicores (milli-cores)
RAM — in bytes
For CPU resources, cores are used. For example, if we need to allocate one physical CPU core to a container, the manifest should specify 1.0. To allocate half a core, specify 0.5. A core can be logically divided into millicores, so you can allocate, for example, 100m, which means one-thousandth of a core (1 full CPU core contains 1000 millicores).
For RAM, we specify values in bytes. You can use numbers with the suffixes E, P, T, G, M, k. For example, if a container needs to be allocated 1 gigabyte of memory, you should specify 1G. In megabytes, it would be 1024M, in kilobytes, it would be 1048576k, and so on.
The requests and limits parameters are optional; however, it is important to note that if both parameters are not set, the container will be able to run on any available node in the cluster regardless of the free resources and will consume as many resources as are physically available on each node. Essentially, the cluster will allocate excess resources. This practice can negatively affect the stability of the entire cluster, as it significantly increases the risk of errors such as OOM (Out of Memory) and OutOfCPU (lack of CPU resources). To prevent these errors, Kubernetes introduced the request and limit mechanisms.
Practical Use of Requests and Limits in Kubernetes
Let's look at the practical use of requests and limits. First, we will deploy a deployment file with an Nginx image where we will set only the requests.
In the configuration below, to launch a pod with a container, the node must have at least 100 millicores of CPU (1/1000 of a CPU core) and 150 megabytes of free memory:
apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx-test-deployment
namespace: ns-for-nginx
labels:
app: nginx-test
spec:
selector:
matchLabels:
app: nginx-test
template:
metadata:
labels:
app: nginx-test
spec:
containers:
- name: nginx-test
image: nginx:1.25
resources:
requests:
cpu: "100m"
memory: "150Mi"
Before deploying the deployment, let's create a new namespace named ns-for-nginx:
kubectl create ns ns-for-nginx
After creating the namespace, we will deploy the deployment file using the following command:
kubectl apply -f nginx-test-deployment.yml
Now, let's check if the deployment was successfully created:
kubectl get deployments -A
Also, check the status of the pod:
kubectl get po -n ns-for-nginx
The deployment file and the pod have been successfully launched. To ensure that the minimum resource request was set for the Nginx pod, we will use the kubectl describe pod command (where nginx-test-deployment-786d6fcb57-7kddf is the name of the running pod):
kubectl describe pod nginx-test-deployment-786d6fcb57-7kddf -n ns-for-nginx
In the output of this command, you can find the requests block, which contains the previously set minimum requirements for our container to run:
In the example above, we created a deployment that sets only the minimum required resources for deployment. Now, let's add limits for the container to run with 1 full CPU core and 1 gigabyte of RAM by creating a new deployment file:
apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx-test-deployment-2
namespace: ns-for-nginx
labels:
app: nginx-test2
spec:
selector:
matchLabels:
app: nginx-test2
template:
metadata:
labels:
app: nginx-test2
spec:
containers:
- name: nginx-test2
image: nginx:1.25
resources:
requests:
cpu: "100m"
memory: "150Mi"
limits:
cpu: "1.0"
memory: "1G"
Let's create the deployment in the cluster:
kubectl apply -f nginx-test-deployment2.yml
Using the kubectl describe command, let's verify that both requests and limits have been applied (where nginx-test-deployment-2-6d5df6c95c-brw8n is the name of the pod):
kubectl describe pod nginx-test-deployment-2-6d5df6c95c-brw8n -n ns-for-nginx
In the screenshot above, both requests and limits have been set for the container. With these quotas, the container will be scheduled on a node with at least 150 megabytes of RAM and 100 milli-CPU. At the same time, the container will not be allowed to consume more than 1 gigabyte of RAM and 1 CPU core.
Using ResourceQuota
In addition to manually assigning resources for each container, Kubernetes provides a way to allocate quotas to specific namespaces in the cluster. The ResourceQuota mechanism allows setting resource usage limits within a particular namespace. ResourceQuota is intended to limit resources such as CPU and memory.
The practical use of ResourceQuota looks like this:
Create a new namespace with quota settings:
kubectl create ns ns-for-resource-quota
Create a ResourceQuota object:
apiVersion: v1
kind: ResourceQuota
metadata:
name: resource-quota-test
namespace: ns-for-resource-quota
spec:
hard:
pods: "2"
requests.cpu: "0.5"
requests.memory: "800Mi"
limits.cpu: "1"
limits.memory: "1G"
In this example, for all objects created in the ns-for-resource-quota namespace, the following limits will apply:
A maximum of 2 pods can be created.
The minimum CPU resources required for starting the pods is 0.5 milliCPU.
The minimum memory required for starting the pods is 800MB.
CPU limits are set to 1 core (no more can be allocated).
Memory limits are set to 1GB (no more can be allocated).
Apply the configuration file:
kubectl apply -f test-resource-quota.yaml
Check the properties of the ResourceQuota object:
kubectl get resourcequota resource-quota-test -n ns-for-resource-quota
As you can see, resource quotas have been set.
Also, verify the output of the kubectl describe ns command:
kubectl describe ns ns-for-resource-quota
The previously created namespace ns-for-resource-quota will have the corresponding resource quotas.
Example of an Nginx pod with the following configuration:
apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx-with-quota
namespace: ns-for-resource-quota
labels:
app: nginx-with-quota
spec:
selector:
matchLabels:
app: nginx-with-quota
replicas: 3
template:
metadata:
labels:
app: nginx-with-quota
spec:
containers:
- name: nginx
image: nginx:1.22.1
resources:
requests:
cpu: 100m
memory: 100Mi
limits:
cpu: 100m
memory: 100Mi
Here we define 3 replicas of the Nginx pod to test the quota mechanism. We also set minimum resource requests for the containers and apply limits to ensure the containers don't exceed the defined resources.
Apply the configuration file:
kubectl apply -f nginx-deployment-with-quota.yaml
kubectl get all -n ns-for-resource-quota
As a result, only two of the three replicas of the pod will be successfully created. The deployment will show an error message indicating that the resource quota for pod creation has been exceeded (in this case, we're trying to create more pods than allowed):
However, the remaining two Nginx pods were successfully started:
Conclusion
Requests and limits are critical mechanisms in Kubernetes that allow for flexible resource allocation and control within the cluster, preventing unexpected errors in running applications and ensuring the stability of the cluster itself.
We offer an affordable Kubernetes hosting platform, with transparent and scalable pricing for all workloads.
29 January 2025 · 9 min to read
Kubernetes
How to Deploy PostgreSQL on Kubernetes
PostgreSQL is a popular relational database management system (RDBMS) that provides high-availability features like streaming replication, logical replication, and failover solutions. Deploying PostgreSQL on Kubernetes allows organizations to build resilient systems that ensure minimal downtime and data availability.
With Kubernetes StatefulSets, you can scale PostgreSQL deployment in response to demand.
Kubernetes Environment Setup
To get started, make sure you have the following:
Kubernetes Cluster (Cloud or Local): You can set up a Kubernetes cluster on Hostman within no time. To follow this tutorial with a local Kubernetes cluster, you can use one of these tools: k3s, minikube, microk8s, kind.
Kubectl: Kubectl allows users to interact with a Kubernetes cluster. The kubectl needs a configuration YAML file which contains cluster details and is usually provided by your cloud provider.
From the Hostman control panel, you can simply download this configuration file with a click of a button as indicated in the below screenshot.
To connect, you need to set KUBECONFIG environment variable accordingly.
export KUBECONFIG=/absolute/path/to/file/k8s-cluster-config.yaml
Helm: You need Helm CLI to install Helm charts. Helm version 3 is required.
Deploy PostgreSQL Using a Helm Chart
Helm is a package manager for Kubernetes just like apt for Ubuntu and Debian. Instead of manually creating multiple YAML files for Pods, Services, Persistent Volumes, Secrets, etc., the Helm chart simplifies this to a single command (e.g., helm install), streamlining the deployment process.
Step 1: Add helm repository
To add the Bitnami PostgreSQL Helm repo, run this command:
helm repo add bitnami https://charts.bitnami.com/bitnami
To sync your local Helm repository with the remote one:
helm repo update
Step 2: Manage Data Persistence
PostgreSQL requires persistent storage to ensure that data is preserved even if a pod crashes or is rescheduled.
When a Persistent Volume Claim (PVC) is combined with a Persistent Volume (PV), Kubernetes can allocate a desired chunk of storage either in disk or cloud storage. PVC requests the Kubernetes cluster for storage space. Kubernetes then looks at the available PVs and assigns one to it.
Create a file named postgres-local-pv.yaml with the YAML manifest:
apiVersion: v1
kind: PersistentVolume
metadata:
name: postgresql-local-pv
spec:
capacity:
storage: 5Gi
accessModes:
- ReadWriteOnce
persistentVolumeReclaimPolicy: Retain
storageClassName: manual
hostPath:
path: /mnt/data/postgresql
This manifest creates a PersistentVolume backed by a local directory (/mnt/data/postgresql) on a specific node. This means if the node goes down or becomes unavailable, the data stored in that PV will be inaccessible, which is a critical risk in production. Therefore, it’s highly recommended to use cloud-native storage solutions instead of hostPath to ensure reliability, scalability and data protection.
This PV has a reclaim policy of Retain, ensuring that it is not deleted when no longer in use by a PVC.
You can set storageClassName to ceph-storage, glusterfs, portworx-sc, or openebs-standard based on your needs.
Create a file named postgres-local-pvc.yaml with this text:
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: postgresql-local-pvc
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 5Gi
storageClassName: manual
The ReadWriteOnce config means the volume can be read-write by a single node at a time. You might think, replacing it with ReadWriteMany will make your application highly available. This isn’t the case.
ReadWriteMany (RWX) access mode allows multiple pods to access the same PersistentVolume simultaneously, this can indeed create serious issues leading to potential race conditions, data corruption, or inconsistent state.
Apply these manifests using kubectl and create new resources.
kubectl apply -f postgres-local-pv.yamlkubectl apply -f postgres-local-pvc.yaml
Step 3: Install PostgreSQL Helm Chart
Run the following command to install the Helm chart.
helm install tutorial-db bitnami/postgresql --set auth.username=bhuwan \
--set auth.password=”AeSeigh2gieshe” \
--set auth.database=k8s-tutorial \
--set auth.postgresPassword=”Ze4hahshez6dop9vaing” \
--set primary.persistence.existingClaim=postgresql-local-pvc \
--set volumePermissions.enabled=true
After a couple of minutes, verify if things have worked successfully with this command:
kubectl get all
Step 4: Test and Connect
The following command runs a temporary PostgreSQL client pod. The pod connects to the database named k8s-tutorial, using the username bhuwan and the password from the environment variable $POSTGRES_PASSWORD.
export POSTGRES_PASSWORD=$(kubectl get secret --namespace default tutorial-db-postgresql -o jsonpath="{.data.password}" | base64 -d)
kubectl run tutorial-db-postgresql-client --rm --tty -i --restart='Never' \
--image docker.io/bitnami/postgresql:17.2.0-debian-12-r6 \
--env="PGPASSWORD=$POSTGRES_PASSWORD" \
--command -- psql --host tutorial-db-postgresql \
-U bhuwan -d k8s-tutorial -p 5432
After the session ends, the pod will be deleted automatically due to the --rm flag.
A quick reminder, if you have changed the Helm chart release name, users, or database name, adjust the above commands accordingly.
Deploy Postgres on Kubernetes from scratch
A StatefulSet is the best Kubernetes resource for deploying stateful applications like PostgreSQL. This way, every PostgreSQL pod gets its own stable network identities and persistent volumes.
Note: you’ll be using a previously created Persistent Volume Claim (PVC) and Persistent Volume(PV). So, do some cleanup and recreate those resources.
helm delete tutorial-db
kubectl delete pvc postgresql-local-pvc
kubectl delete pv postgresql-local-pv
kubectl apply -f postgres-local-pv.yaml -f postgres-local-pvc.yaml
Create a file named postgres-statefulset.yaml with the following text:
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: postgres-statefulset
labels:
app: postgres
spec:
serviceName: "postgresql-headless-svc"
replicas: 1
selector:
matchLabels:
app: postgres
template:
metadata:
labels:
app: postgres
spec:
containers:
- name: postgres
image: postgres:17.2
envFrom:
- secretRef:
name: postgresql-secret
ports:
- containerPort: 5432
name: postgresdb
volumeMounts:
- name: pv-data
mountPath: /var/lib/postgresql/db
volumes:
- name: pv-data
persistentVolumeClaim:
claimName: postgresql-local-pvc
Before you can apply these changes, create a new Secret for handling sensitive details like passwords with kubectl.
kubectl create secret generic postgresql-secret --from-literal=POSTGRES_PASSWORD=Ze4hahshez6dop9vaing
kubectl apply -f postgres-statefulset.yaml
If the pod gets stuck with Pending state, you can try creating a StorageClass with the following manifest.
kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
name: manual
provisioner: kubernetes.io/no-provisioner
volumeBindingMode: WaitForFirstConsumer
To investigate any further issues with the pod, you can use the command:
kubectl describe pod postgres-statefulset-0
This command will report any issues related to scheduling the pod to a node, mounting volumes, or resource constraints.
Databases like PostgreSQL are typically accessed internally by other services or applications within the cluster, so it's better to create a Headless service for it.
Create a file called postgres-service.yaml and include the following YAML manifest:
apiVersion: v1
kind: Service
metadata:
name: postgresql-headless-svc
spec:
type: ClusterIP
selector:
app: postgres
ports:
- port: 5432
targetPort: 5432
clusterIP: None
Finally, you can test the connection with kubectl run.
kubectl run tutorial-db-postgresql-client --rm --tty -i --restart='Never' \
--image docker.io/bitnami/postgresql:17.2.0-debian-12-r6 \
--env="PGPASSWORD=Ze4hahshez6dop9vaing" \
--command -- psql --host postgres-statefulset-0.postgresql-headless-svc \
-U postgres -p 5432
Scale, Replication, and Backup
To scale up a Statefulset, simply pass the number of replicas with --replicas flag.
kubectl scale statefulset postgres-statefulset --replicas=3
To reach replicas, you can make use of headless service. For instance, with hostname postgres-statefulset-1.postgresql-headless-svc you can send requests to pod 1.
For handling backups, you can use CronJob with the pg_dump utility provided by PostgreSQL.
Best Practices
Throughout the tutorial, the decision to handle passwords via Kubernetes Secret, using StatefulSet instead of Deployment was a good move. To make this deployment even more secure, reliable, and highly available, here are some ideas:
Set Resource Requests and Limits: Set appropriate CPU and memory requests and limits to avoid over-provisioning and under-provisioning.
Backups: Use Kubernetes CronJobs to regularly back up your PostgreSQL data. Consider implementing Volume Snapshots as well.
Monitoring and Log Postgresql: You can use tools like Prometheus and Grafana to collect and visualize PostgreSQL metrics, such as query performance, disk usage, and replication status.
Use Pod Disruption Budgets (PDBs): If too many PostgreSQL pods are disrupted at once (e.g., during a rolling update), it can lead to database unavailability or replication issues.
Conclusion
Helm chart is the recommended way of complex and production deployment. Helm provides an automated version manager alongside hiding the complexities of configuring individual Kubernetes components. Using the Helm template command, you can even render the Helm chart locally and make necessary adjustments with its YAML Kubernetes manifests.
Kubernetes provides scalability, flexibility, and ease of automation for PostgreSQL databases. By leveraging Kubernetes features like StatefulSets, PVCs, PDBs, and secrets management, you can ensure that your PostgreSQL database is tuned for the production environment.
24 January 2025 · 8 min to read
Kubernetes
Installing MongoDB in a Kubernetes Cluster
MongoDB is a widely used NoSQL database designed to store large volumes of unstructured data. Combined with Kubernetes, MongoDB becomes a powerful solution for scaling databases efficiently within a unified environment.
Prerequisites
To install MongoDB on Kubernetes, you'll need a configured cloud server (or a physical one) with superuser rights and a Kubernetes cluster. While any OS can be used, Linux is recommended for minimal installation issues.
Step-by-Step MongoDB Installation
Connect to the Server:
Gain superuser access and install necessary software:
sudo -s
apt-get update && apt install curl apt-transport-https -y && curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | apt-key add - && echo "deb https://apt.kubernetes.io/ kubernetes-xenial main" | tee -a /etc/apt/sources.list.d/kubernetes.list && apt-get update && apt install kubectl -y
Configure Kubernetes Environment:
Create a directory, add the configuration file, and set the environment variable:
mkdir /usr/local/etc/mongo && cd /usr/local/etc/mongo
cat << EOF > testcluster.conf<insert your cluster config data here>EOF
echo "export KUBECONFIG=testcluster.conf" >> ~/.bashrc
Verify Connection:
Use kubectl cluster-info to check the connection. A successful connection will display:
Kubernetes control plane is running at <IP>.
Create MongoDB Configuration Files:
Set up a container for data storage and create a Creds.yaml file for MongoDB credentials. Encrypt login and password using BASE64:
echo <unencrypted data> | base64echo <encrypted data> | base64 -d
Example:
apiVersion: v1
data:
username: <username encrypted with BASE64>
password: <password encrypted with BASE64>
kind: Secret
metadata:
creationTimestamp: null
name: creds
Deploy MongoDB:
Create a PersistVolClaim.yaml file with MongoDB configuration and deploy it using:
kubectl apply -f PersistVolClaim.yaml
The file example:
apiVersion: apps/v1
kind: Deployment
metadata:
labels:
app: mongo
name: mongo
spec:
replicas: 1
selector:
matchLabels:
app: mongo
strategy: {}
template:
metadata:
labels:
app: mongo
spec:
containers:
- image: mongo
name: mongo
args: ["--dbpath","/data/db"]
livenessProbe:
exec:
command:
- mongo
- --disableImplicitSessions
- --eval
readinessProbe:
exec:
command:
- mongo
- --disableImplicitSessions
- --eval
env:
- name: MONGO_INITDB_ROOT_USERNAME
valueFrom:
secretKeyRef:
name: creds
key: username
- name: MONGO_INITDB_ROOT_PASSWORD
valueFrom:
secretKeyRef:
name: creds
key: password
volumeMounts:
- name: "datadir"
mountPath: "/data/db"
volumes:
- name: "datadir"
persistentVolumeClaim:
claimName: "mongopvc"
Test MongoDB Connection:
After deploying containers, verify the connection:
kubectl exec deployment/client -it -- /bin/bashmongo
If everything is connected successfully, the system will display a typical database prompt. To create a new database, simply switch to it; however, note that it will not be saved until you add some data. This can be done as follows:
use database_name
db.createCollection("newdata")
show dbs
The last command is used to verify that the newly created database exists.
Considerations for MongoDB in Kubernetes
Remote Storage: For flexibility, use remote storage for MongoDB to facilitate movement if needed.
Resource Management: Configure requests and limits in replica pods to avoid performance issues.
Pod Disruption Budget: Set up to maintain the desired number of running replicas.
Other Tools and Customization
The method of installing MongoDB in Kubernetes described here is one of many options. You can also use software specifically designed to work with Kubernetes, such as Helm or KubeDB. KubeDB, in particular, was created to simplify the integration of other products into Kubernetes. As for Helm, it is another popular solution by VMware (although VMware didn't develop it but acquired and now maintains the product).
Another solution is Percona Operator. This modern, open-source application (developed in 2018) is user-friendly and continuously improved by the community. Some people use combined solutions like Percona + Helm. However, installing MongoDB using each of these applications has its nuances, so it's advisable to study these products before proceeding; plenty of documentation is available.
In conclusion, you can use a customized MongoDB image to manage a MongoDB cluster in Kubernetes according to your specific needs. For example, the default MongoDB image doesn't include authentication. Therefore, you can download an image with pre-configured authentication or create your own. Of course, using customized Docker images is slightly more complex than the implementation described above. Still, it gives you full control over the database configurations and settings according to your requirements. You can find useful information on customizing the official MongoDB image here.
Conclusion
With this guide, you can deploy MongoDB in a Kubernetes cluster. However, further tasks will require some knowledge of Kubernetes, so if you're not familiar with it, we recommend first studying the official documentation.
23 August 2024 · 5 min to read
Do you have questions,
comments, or concerns?
Our professionals are available to assist you at any moment,
whether you need help or are just unsure of where to start.
whether you need help or are just unsure of where to start.
