Load Balancing in Kubernetes

PostgreSQL is a popular relational database management system (RDBMS) that provides high-availability features like streaming replication, logical replication, and failover solutions. Deploying PostgreSQL on Kubernetes allows organizations to build resilient systems that ensure minimal downtime and data availability. With Kubernetes StatefulSets, you can scale PostgreSQL deployment in response to demand. Kubernetes Environment Setup To get started, make sure you have the following: Kubernetes Cluster (Cloud or Local):  You can set up a Kubernetes cluster on Hostman within no time. To follow this tutorial with a local Kubernetes cluster, you can use one of these tools: k3s, minikube, microk8s, kind. Kubectl: Kubectl allows users to interact with a Kubernetes cluster. The kubectl needs a configuration YAML file which contains cluster details and is usually provided by your cloud provider.  From the Hostman control panel, you can simply download this configuration file with a click of a button as indicated in the below screenshot. To connect, you need to set KUBECONFIG environment variable accordingly. export KUBECONFIG=/absolute/path/to/file/k8s-cluster-config.yaml Helm: You need Helm CLI to install Helm charts. Helm version 3 is required. Deploy PostgreSQL Using a Helm Chart Helm is a package manager for Kubernetes just like apt for Ubuntu and Debian. Instead of manually creating multiple YAML files for Pods, Services, Persistent Volumes, Secrets, etc., the Helm chart simplifies this to a single command (e.g., helm install), streamlining the deployment process. Step 1: Add helm repository To add the Bitnami PostgreSQL Helm repo, run this command: helm repo add bitnami https://charts.bitnami.com/bitnami To sync your local Helm repository with the remote one: helm repo update Step 2: Manage Data Persistence PostgreSQL requires persistent storage to ensure that data is preserved even if a pod crashes or is rescheduled. When a Persistent Volume Claim (PVC) is combined with a Persistent Volume (PV), Kubernetes can allocate a desired chunk of storage either in disk or cloud storage. PVC requests the Kubernetes cluster for storage space. Kubernetes then looks at the available PVs and assigns one to it. Create a file named postgres-local-pv.yaml with the YAML manifest: apiVersion: v1 kind: PersistentVolume metadata: name: postgresql-local-pv spec: capacity: storage: 5Gi accessModes: - ReadWriteOnce persistentVolumeReclaimPolicy: Retain storageClassName: manual hostPath: path: /mnt/data/postgresql This manifest creates a PersistentVolume backed by a local directory (/mnt/data/postgresql) on a specific node. This means if the node goes down or becomes unavailable, the data stored in that PV will be inaccessible, which is a critical risk in production. Therefore, it’s highly recommended to use cloud-native storage solutions instead of hostPath to ensure reliability, scalability and data protection. This PV has a reclaim policy of Retain, ensuring that it is not deleted when no longer in use by a PVC. You can set storageClassName to ceph-storage, glusterfs, portworx-sc, or openebs-standard based on your needs. Create a file named postgres-local-pvc.yaml with this text: apiVersion: v1 kind: PersistentVolumeClaim metadata: name: postgresql-local-pvc spec: accessModes: - ReadWriteOnce resources: requests: storage: 5Gi storageClassName: manual The ReadWriteOnce config means the volume can be read-write by a single node at a time. You might think, replacing it with ReadWriteMany will make your application highly available. This isn’t the case. ReadWriteMany (RWX) access mode allows multiple pods to access the same PersistentVolume simultaneously, this can indeed create serious issues leading to potential race conditions, data corruption, or inconsistent state. Apply these manifests using kubectl and create new resources. kubectl apply -f postgres-local-pv.yamlkubectl apply -f postgres-local-pvc.yaml Step 3: Install PostgreSQL Helm Chart Run the following command to install the Helm chart. helm install tutorial-db bitnami/postgresql --set auth.username=bhuwan \ --set auth.password=”AeSeigh2gieshe” \ --set auth.database=k8s-tutorial \ --set auth.postgresPassword=”Ze4hahshez6dop9vaing” \ --set primary.persistence.existingClaim=postgresql-local-pvc \ --set volumePermissions.enabled=true After a couple of minutes, verify if things have worked successfully with this command: kubectl get all Step 4: Test and Connect The following command runs a temporary PostgreSQL client pod. The pod connects to the database named k8s-tutorial, using the username bhuwan and the password from the environment variable $POSTGRES_PASSWORD. export POSTGRES_PASSWORD=$(kubectl get secret --namespace default tutorial-db-postgresql -o jsonpath="{.data.password}" | base64 -d) kubectl run tutorial-db-postgresql-client --rm --tty -i --restart='Never' \ --image docker.io/bitnami/postgresql:17.2.0-debian-12-r6 \ --env="PGPASSWORD=$POSTGRES_PASSWORD" \ --command -- psql --host tutorial-db-postgresql \ -U bhuwan -d k8s-tutorial -p 5432 After the session ends, the pod will be deleted automatically due to the --rm flag. A quick reminder, if you have changed the Helm chart release name, users, or database name, adjust the above commands accordingly. Deploy Postgres on Kubernetes from scratch A StatefulSet is the best Kubernetes resource for deploying stateful applications like PostgreSQL. This way, every PostgreSQL pod gets its own stable network identities and persistent volumes. Note: you’ll be using a previously created Persistent Volume Claim (PVC) and Persistent Volume(PV). So, do some cleanup and recreate those resources. helm delete tutorial-db kubectl delete pvc postgresql-local-pvc kubectl delete pv postgresql-local-pv kubectl apply -f postgres-local-pv.yaml -f postgres-local-pvc.yaml Create a file named postgres-statefulset.yaml with the following text: apiVersion: apps/v1 kind: StatefulSet metadata: name: postgres-statefulset labels: app: postgres spec: serviceName: "postgresql-headless-svc" replicas: 1 selector: matchLabels: app: postgres template: metadata: labels: app: postgres spec: containers: - name: postgres image: postgres:17.2 envFrom: - secretRef: name: postgresql-secret ports: - containerPort: 5432 name: postgresdb volumeMounts: - name: pv-data mountPath: /var/lib/postgresql/db volumes: - name: pv-data persistentVolumeClaim: claimName: postgresql-local-pvc Before you can apply these changes, create a new Secret for handling sensitive details like passwords with kubectl. kubectl create secret generic postgresql-secret --from-literal=POSTGRES_PASSWORD=Ze4hahshez6dop9vaing kubectl apply -f postgres-statefulset.yaml If the pod gets stuck with Pending state, you can try creating a StorageClass with the following manifest. kind: StorageClass apiVersion: storage.k8s.io/v1 metadata: name: manual provisioner: kubernetes.io/no-provisioner volumeBindingMode: WaitForFirstConsumer To investigate any further issues with the pod, you can use the command: kubectl describe pod postgres-statefulset-0 This command will report any issues related to scheduling the pod to a node, mounting volumes, or resource constraints. Databases like PostgreSQL are typically accessed internally by other services or applications within the cluster, so it's better to create a Headless service for it. Create a file called postgres-service.yaml and include the following YAML manifest: apiVersion: v1 kind: Service metadata: name: postgresql-headless-svc spec: type: ClusterIP selector: app: postgres ports: - port: 5432 targetPort: 5432 clusterIP: None Finally, you can test the connection with kubectl run. kubectl run tutorial-db-postgresql-client --rm --tty -i --restart='Never' \ --image docker.io/bitnami/postgresql:17.2.0-debian-12-r6 \ --env="PGPASSWORD=Ze4hahshez6dop9vaing" \ --command -- psql --host postgres-statefulset-0.postgresql-headless-svc \ -U postgres -p 5432 Scale, Replication, and Backup To scale up a Statefulset, simply pass the number of replicas with --replicas flag.  kubectl scale statefulset postgres-statefulset --replicas=3  To reach replicas, you can make use of headless service. For instance, with hostname postgres-statefulset-1.postgresql-headless-svc you can send requests to pod 1. For handling backups, you can use CronJob with the pg_dump utility provided by PostgreSQL. Best Practices Throughout the tutorial, the decision to handle passwords via Kubernetes Secret, using StatefulSet instead of Deployment was a good move. To make this deployment even more secure, reliable, and highly available, here are some ideas: Set Resource Requests and Limits: Set appropriate CPU and memory requests and limits to avoid over-provisioning and under-provisioning. Backups: Use Kubernetes CronJobs to regularly back up your PostgreSQL data. Consider implementing Volume Snapshots as well. Monitoring and Log Postgresql: You can use tools like Prometheus and Grafana to collect and visualize PostgreSQL metrics, such as query performance, disk usage, and replication status. Use Pod Disruption Budgets (PDBs): If too many PostgreSQL pods are disrupted at once (e.g., during a rolling update), it can lead to database unavailability or replication issues. Conclusion Helm chart is the recommended way of complex and production deployment. Helm provides an automated version manager alongside hiding the complexities of configuring individual Kubernetes components. Using the Helm template command, you can even render the Helm chart locally and make necessary adjustments with its YAML Kubernetes manifests. Kubernetes provides scalability, flexibility, and ease of automation for PostgreSQL databases. By leveraging Kubernetes features like StatefulSets, PVCs, PDBs, and secrets management, you can ensure that your PostgreSQL database is tuned for the production environment.

MongoDB is a widely used NoSQL database designed to store large volumes of unstructured data. Combined with Kubernetes, MongoDB becomes a powerful solution for scaling databases efficiently within a unified environment. Prerequisites To install MongoDB on Kubernetes, you'll need a configured cloud server (or a physical one) with superuser rights and a Kubernetes cluster. While any OS can be used, Linux is recommended for minimal installation issues. Step-by-Step MongoDB Installation Connect to the Server: Gain superuser access and install necessary software: sudo -s apt-get update && apt install curl apt-transport-https -y && curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | apt-key add - && echo "deb https://apt.kubernetes.io/ kubernetes-xenial main" | tee -a /etc/apt/sources.list.d/kubernetes.list && apt-get update && apt install kubectl -y Configure Kubernetes Environment: Create a directory, add the configuration file, and set the environment variable: mkdir /usr/local/etc/mongo && cd /usr/local/etc/mongo cat << EOF > testcluster.conf<insert your cluster config data here>EOF echo "export KUBECONFIG=testcluster.conf" >> ~/.bashrc Verify Connection: Use kubectl cluster-info to check the connection. A successful connection will display:  Kubernetes control plane is running at <IP>. Create MongoDB Configuration Files: Set up a container for data storage and create a Creds.yaml file for MongoDB credentials. Encrypt login and password using BASE64: echo <unencrypted data> | base64echo <encrypted data> | base64 -d Example: apiVersion: v1 data: username: <username encrypted with BASE64> password: <password encrypted with BASE64> kind: Secret metadata: creationTimestamp: null name: creds Deploy MongoDB: Create a PersistVolClaim.yaml file with MongoDB configuration and deploy it using: kubectl apply -f PersistVolClaim.yaml The file example: apiVersion: apps/v1 kind: Deployment metadata: labels: app: mongo name: mongo spec: replicas: 1 selector: matchLabels: app: mongo strategy: {} template: metadata: labels: app: mongo spec: containers: - image: mongo name: mongo args: ["--dbpath","/data/db"] livenessProbe: exec: command: - mongo - --disableImplicitSessions - --eval readinessProbe: exec: command: - mongo - --disableImplicitSessions - --eval env: - name: MONGO_INITDB_ROOT_USERNAME valueFrom: secretKeyRef: name: creds key: username - name: MONGO_INITDB_ROOT_PASSWORD valueFrom: secretKeyRef: name: creds key: password volumeMounts: - name: "datadir" mountPath: "/data/db" volumes: - name: "datadir" persistentVolumeClaim: claimName: "mongopvc" Test MongoDB Connection: After deploying containers, verify the connection: kubectl exec deployment/client -it -- /bin/bashmongo If everything is connected successfully, the system will display a typical database prompt. To create a new database, simply switch to it; however, note that it will not be saved until you add some data. This can be done as follows: use database_name db.createCollection("newdata") show dbs The last command is used to verify that the newly created database exists. Considerations for MongoDB in Kubernetes Remote Storage: For flexibility, use remote storage for MongoDB to facilitate movement if needed. Resource Management: Configure requests and limits in replica pods to avoid performance issues. Pod Disruption Budget: Set up to maintain the desired number of running replicas. Other Tools and Customization The method of installing MongoDB in Kubernetes described here is one of many options. You can also use software specifically designed to work with Kubernetes, such as Helm or KubeDB. KubeDB, in particular, was created to simplify the integration of other products into Kubernetes. As for Helm, it is another popular solution by VMware (although VMware didn't develop it but acquired and now maintains the product). Another solution is Percona Operator. This modern, open-source application (developed in 2018) is user-friendly and continuously improved by the community. Some people use combined solutions like Percona + Helm. However, installing MongoDB using each of these applications has its nuances, so it's advisable to study these products before proceeding; plenty of documentation is available. In conclusion, you can use a customized MongoDB image to manage a MongoDB cluster in Kubernetes according to your specific needs. For example, the default MongoDB image doesn't include authentication. Therefore, you can download an image with pre-configured authentication or create your own. Of course, using customized Docker images is slightly more complex than the implementation described above. Still, it gives you full control over the database configurations and settings according to your requirements. You can find useful information on customizing the official MongoDB image here. Conclusion With this guide, you can deploy MongoDB in a Kubernetes cluster. However, further tasks will require some knowledge of Kubernetes, so if you're not familiar with it, we recommend first studying the official documentation.

Kubernetes, or K8s, is an open-source container orchestration platform developed by Google. The core concept behind Kubernetes is that a user installs it on a server, or more likely a cluster, and deploys various workloads on it. Kubernetes addresses challenges related to container creation, scaling, namespaces, access rights, and more. The primary interaction with the cluster is through YAML configuration files. This tutorial will guide you through creating and deploying a Kubernetes cluster locally. Creating Virtual Machines We will set up the Kubernetes cluster on two virtual machines: one acting as the master node and the other as a worker node. While deploying a cluster with only two nodes is not practical for real-world use, it is sufficient for educational purposes. If you wish to create a Kubernetes cluster with more nodes, simply repeat the process for each additional node. We will use Oracle's VirtualBox to create virtual machines, which you can download from this link. After installation, proceed to create the virtual machines. For the operating system, we will use Ubuntu Server, which can be downloaded here. After downloading, open VirtualBox. Click "Create" in VirtualBox to create a new virtual machine. The default settings are sufficient, but allocate 3 GB of RAM and 2 CPUs for the master node (which manages the Kubernetes cluster) and 2 GB of RAM for the worker node. Kubernetes requires a minimum of 2 CPUs for the master node. Create two virtual machines this way. After creating the virtual machines, create a boot image with the Ubuntu Server distribution. Go to "Storage" and click "Choose/Create a Disk Image." Click "Add" and select the Ubuntu Server distribution. Then, start both machines and install the operating system by selecting "Try or Install Ubuntu." During installation, create users for each system and choose the default settings. After installation, shut down both virtual machines and go to their settings. In the "Network" section, change the connection type to "Bridged Adapter" for each system so that the virtual machines can communicate with each other over the network. System Preparation Network Configuration Set the node names for the cluster. On the master node, execute the following command: sudo hostnamectl set-hostname master.local On the worker node, execute: sudo hostnamectl set-hostname worker.local If there are multiple worker nodes, assign each a unique name: worker1.local, worker2.local, and so on. To ensure that nodes are accessible by name, modify the hosts file on each node. Add the following lines: 192.168.43.80     master.local master192.168.43.77     worker.local worker Here, 192.168.43.80 and 192.168.43.77 are the IP addresses of each node. To find the IP address, use the ip addr command: ip addr Locate the IP address next to inet. Open the hosts file and make the necessary edits: sudo nano /etc/hosts To verify that the VMs can communicate with each other, ping the nodes: ping 192.168.43.80 If successful, you will receive a response similar to this: PING 192.168.43.80 (192.168.43.80) 56(84) bytes of data.64 bytes from 192.168.43.80: icmp_seq=1 ttl=64 time=0.054 ms Updating Packages and Installing Additional Utilities Next, install the necessary utilities and packages on each node. These steps should be applied to each node unless specified otherwise. Start by updating the package list and systems: sudo apt-get update && apt-get upgrade -y Then install the following packages: sudo apt-get install curl apt-transport-https git iptables-persistent -y Swap File Kubernetes will not start with an active swap file, so it needs to be disabled: sudo swapoff -a To prevent it from reactivating after a reboot, modify the fstab file: sudo nano /etc/fstab Comment out the line with #: # /swap.img      none    swap    sw      0       0 Kernel Configuration Load additional kernel modules: sudo nano /etc/modules-load.d/k8s.conf Add the following two lines to k8s.conf: br_netfilteroverlay Now, load the modules into the kernel: sudo modprobe br_netfiltersudo modprobe overlay Verify the modules are loaded successfully: sudo lsmod | egrep "br_netfilter|overlay" You should see output similar to this: overlay               147456  0br_netfilter           28672  0bridge                299008  1 br_netfilter Create a configuration file to process traffic through the bridge in netfilter: sudo nano /etc/sysctl.d/k8s.conf Add the following two lines: net.bridge.bridge-nf-call-ip6tables = 1net.bridge.bridge-nf-call-iptables = 1 Apply the settings: sudo sysctl --system Docker Installation Run the following command to install Docker: sudo apt-get install docker docker.io -y For more details on installing Docker on Ubuntu, refer to the official guide. After installation, enable Docker to start on boot and restart the service: sudo systemctl enable dockersudo systemctl restart docker Kubernetes Installation Add the GPG key: sudo curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo apt-key add - Next, create a repository configuration file: sudo nano /etc/apt/sources.list.d/kubernetes.list Add the following entry: deb https://apt.kubernetes.io/ kubernetes-xenial main Update the apt-get package list: sudo apt-get update Install the following packages: sudo apt-get install kubelet kubeadm kubectl Installation is now complete. Verify the Kubernetes client version: sudo kubectl version --client  The output should be similar to this: Client Version: version.Info{Major:"1", Minor:"24", GitVersion:"v1.24.2"} Cluster Configuration Master Node Run the following command for the initial setup and preparation of the master node: sudo kubeadm init --pod-network-cidr=10.244.0.0/16 The --pod-network-cidr flag specifies the internal subnet address, with 10.244.0.0/16 being the default value. The process will take a few minutes. Upon completion, you will see the following message: Then you can join any number of worker nodes by running the following on each as root:kubeadm join 192.168.43.80:6443 --token f7sihu.wmgzwxkvbr8500al \--discovery-token-ca-cert-hash sha256:6746f66b2197ef496192c9e240b31275747734cf74057e04409c33b1ad280321 Save this command to connect the worker nodes to the master node. Create the KUBECONFIG environment variable: export KUBECONFIG=/etc/kubernetes/admin.conf Install the Container Network Interface (CNI): kubectl apply -f https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml Worker Node On the worker node, run the kubeadm join command obtained during the master node setup. After this, on the master node, enter: sudo kubectl get nodes The output should be: NAME                 STATUS      ROLES                        AGE    VERSIONmaster.local          Ready      control-plane,master          10m    v1.24.2worker.local          Ready      <none>                        79s    v1.24.2 The cluster is now deployed and ready for operation. Conclusion Setting up a Kubernetes cluster involves several steps, from creating and configuring virtual machines to installing and configuring the necessary software components. This tutorial provided a step-by-step guide to deploying a basic Kubernetes cluster on a local environment. While this setup is suitable for educational purposes, real-world deployments typically involve more nodes and more complex configurations. Kubernetes provides powerful tools for managing containerized applications, making it a valuable skill for modern IT professionals. By following this guide, you've taken the first steps in mastering Kubernetes and its ecosystem.