Managing Multiple Kubernetes Clusters Using Git

Technical writer

Kubernetes

15.08.2024

Reading time: 5 min

One of the key decisions for companies utilizing cloud architecture is how to manage multiple Kubernetes clusters. Developers often choose GitHub due to its popularity and familiarity among IT professionals. However, if you need truly efficient management of multiple Kubernetes clusters, it's better to opt for its main competitor—GitLab. Let us explain why GitLab is more suitable for this task.

In this article, we will focus on optimizing cluster management through GitLab and explain which tools can be used to ensure structure and improve cluster interactions through hierarchical groups. We will also cover the advantages of centralizing CI/CD code, achieved through GitLab's extensive template capabilities.

GitLab vs GitHub

Both DevOps tools offer numerous useful features for interacting with Kubernetes functionality, but GitLab is the better choice for managing multiple Kubernetes clusters. GitLab provides efficient management of manifests in Git repositories, allowing you to structure them in a multi-cloud environment.

In this case, the main advantage of GitLab lies in its optimal hierarchical structure, which allows you to define an unlimited number of groups. In contrast, GitHub imposes several restrictions that prevent the definition of hierarchical groups altogether. For this reason, GitLab is the right choice for multi-cloud tasks. Now, let's go through the steps to achieve this.

Step 1: Preparation

Before working with GitLab on our server, you need to deploy and configure it. Remember that the minimum requirements for proper GitLab operation on a server (4 GB RAM and 4 CPU cores) may increase depending on the load on the supported clusters.

Step 2: Creating Hierarchical Groups

GitLab allows you to define groups within other groups, inheriting all environment variables from the parent groups. First, you need to define a kubeconfig file in the parent group, after which you can use the environment variables in all associated projects. For example, consider Kubernetes clusters in different environments, such as Google Cloud, Azure, and others. First, create a group and project hierarchy.

Each group represents a specific Kubernetes cluster, and each set contains manifest files related to specific projects. For instance, you can create a project named api-gateway within the kube-ny-1 group. This way, the manifests within the project will be intended for deployment in our Kubernetes cluster, located in the ny-1 region (the region is just an example and can be different).

Additionally, to view Kubernetes clusters at the group level, select Search or go to from the left sidebar. Then choose Operate → Kubernetes.

Step 3: Placing KubeConfig in Environment Variables at the Group Level

Each project in a GitLab group can be configured to access common environment variables set in both the parent group and higher levels. To do this, you need to correctly configure the Kubeconfig file, which contains the credentials for connecting to the Kubernetes API server. This is done in the group settings via the following path: Group Name → CI/CD Settings → Variables → Add Variable. Example:

^{Image source: itnext.io}

Thus, we need to place the Kubeconfig files of all our Kubernetes clusters into the groups that correspond to them.

Step 4: Deploying the Application Using GitLab CI

Now, within each project, create manifest files and deploy them using the GitLab CI pipeline. Here's how it's done (using our example api-gateway in the kube-ny-1 group):

.
├── gitlab-ci.yml
└── manifests
   ├── configmap.yaml
   ├── deployment.yaml
   └── service.yaml

You can deploy your manifests within the gitlab-ci.yml file like this:

stages:
 - deploy

deploy:
 stage: deploy
 image: bitnami/kubectl
 script: |
   for f in manifests/*
   do
       kubectl apply -f $f
   done

Note that we define the Kubeconfig environment variable in the parent group, so there's no need to define it for each project.

Step 5: Assigning Access

Of course, GitLab allows you to delegate management of your groups and projects to team members, and this can be done according to the hierarchy. When you assign access at the group level, all projects within the group will inherit a unified access model, which is very convenient as it eliminates the need to change access settings for each project. Here's what granting Maintainer-level access to all projects within a group looks like:

^{Image source: itnext.io}

Step 6: Encapsulating CI/CD Code

The template creation feature is incredibly useful for us here—another trump card for GitLab compared to its main competitor. Define a common template for your pipeline and include it wherever necessary. Now, if you need to modify the code, the changes will be applied globally, not in each individual project.

To encapsulate CI/CD code, create a project to store it, then include it in other projects. Inside the template project, there should be an install.yml file with content corresponding to the gitlab-ci.yml file (see the code example in Step 3). Now, update all your projects and place the updated data in the gitlab-ci.yml file, like this:

include:
 - project: 'cd/template'
   ref: 'main'
   file: 'install.yml'

That's it! You can consider your Kubernetes cluster group setup complete.

Conclusion

In this article, we explored how to manage multiple Kubernetes clusters using GitLab. We have learned how to create hierarchical groups in GitLab, configure Kubeconfig at the group level, deploy an application using GitLab CI, assign user access, and encapsulate CI/CD code to automate project management.

Kubernetes

15.08.2024

Reading time: 5 min

Similar

Kubernetes

Managing Multiple Kubernetes Clusters Using Git

GitLab vs GitHub

Step 1: Preparation

Step 2: Creating Hierarchical Groups

Step 3: Placing KubeConfig in Environment Variables at the Group Level

Step 4: Deploying the Application Using GitLab CI

Step 5: Assigning Access

Step 6: Encapsulating CI/CD Code

Conclusion

Similar

Installing MongoDB in a Kubernetes Cluster

Kubernetes Cluster: Installation, Configuration, and Management

Running Kubernetes Clusters in the Cloud with VMware

Do you have questions, comments, or concerns?

Do you have questions,
comments, or concerns?