Restricting Access in Kubernetes
Kubernetes
19.08.2024
Reading time: 6 min
Share
Deploying a Kubernetes cluster is relatively easy, even for beginners. However, maintaining its functionality is a different story. One of the key tasks here is managing access rights to prevent cluster issues. In this guide, we'll explore the most effective way to restrict access, minimizing the chances of cluster disruptions due to accidental configuration changes by inexperienced users. But first, let's cover some basics.
How Access Control Works in Kubernetes
Kubernetes access control is based on the concept of roles and permissions, known as Role-Based Access Control (RBAC). RBAC allows Kubernetes administrators to define who has access to which resources and operations within the cluster.
The following key entities are used for configuring Role-Based Access Control in Kubernetes:
-
Roles: Define what actions are permitted on specific resources (e.g., read, write, delete).
-
RoleBindings: Link roles to specific users, service accounts, or groups.
-
ServiceAccounts: Used to authenticate applications and services within the cluster.
With RBAC, administrators can control access to various Kubernetes resources, such as pods, services, and storage, based on the needs and roles of users or services. RBAC provides a flexible access management system that helps ensure security and control over the cluster.
RBAC allows you to set access controls at the Kubernetes cluster level (using ClusterRole and ClusterRoleBinding) or limit them within a specific namespace (using Role and RoleBinding).
Creating Roles and RoleBindings to Restrict Access
To create a Role and RoleBinding in Kubernetes, you need to create YAML files defining these objects. Below are specific examples. First, here's a code example for defining a Role (let's call it getlistwatch.yaml):
kind: Role
metadata:
namespace: default
name: getlistwatch
rules:
- apiGroups: [""]
resources: ["pods"]
verbs: ["get", "list", "watch"]Now, here's an example for defining a RoleBinding (let's call it getlistwatch-bind.yaml):
kind: RoleBinding
metadata:
name: getlistwatch-bind
namespace: default
subjects:
- kind: User
name: username # Replace 'username' with the actual user's name
apiGroup: rbac.authorization.k8s.io
roleRef:
kind: Role
name: getlistwatch
apiGroup: rbac.authorization.k8s.ioYou can apply these objects using the kubectl apply -f command. For our examples, it would look like this:
kubectl apply -f getlistwatch.yaml
kubectl apply -f getlistwatch-bind.yaml
In these examples, we created a Role named getlistwatch, which allows getting, listing, and watching pod resources in the cluster. We then created a RoleBinding named getlistwatch-bind, which links this role to a specific user. After applying these files, the user will be granted permission to perform the specified operations on pod resources in the cluster.
It's worth noting that this user will not be able to perform any other actions in the cluster unless other roles are assigned to them, which should be checked separately.
Authentication Methods
There are three primary methods:
-
Basic authentication with configuration passed through the API.
-
Client certificate authentication, certified by the Kubernetes certification authority.
-
Authentication through Bearer-token or JWT.
The first method is rarely used today, so let's move on to the second.
Kubernetes Certificate Authentication
In Kubernetes certificate authentication, each user or service receives its own certificate, which is used for authentication when attempting to access the Kubernetes cluster. The process usually involves the following steps:
-
Generating Certificates: The cluster administrator generates certificates for each user/service using a certificate authority or certificate creation tools. For example, RSA private keys can be created, followed by certificate signing requests sent to the certification authority.
-
Configuring Authentication: The administrator adds the generated certificates to the Kubernetes configuration, specifying which users/services have access to which resources in the cluster. This is done through kubeconfig, generated for each user/service, with the signed certificate added.
-
Creating Roles: At this stage, a Role is created and then linked to the user/service through a
RoleBinding(as shown in the code above).
Now, when attempting to access the cluster, the user/service must provide their certificate for verification. Kubernetes uses this certificate to authenticate and determine access permissions. If the certificate is successfully verified, the user/service is granted the appropriate permissions to perform operations in the cluster. There are also automation tools for this process, such as bash scripts or Ansible.
This method allows you to create a set of standard roles but introduces the challenge of managing access for numerous users/services and the complexity of certificate revocation. Therefore, in many cases, it's better and safer to use third-party authentication services like DEX and Keycloak, which provide secure authentication via OIDC (OpenID Connect).
Authentication via DEX
One of the key advantages of DEX is its ease of use. However, setting up DEX requires the creation of certificates for both DEX and Gangway, which work in tandem as they communicate through TLS. When deployed within a Kubernetes cluster, entities such as dex.example.com and gangway.example.com will be created, for which certificates are needed. Don't forget to monitor certificate expiration dates programmatically or through cert-manager, as they are time-limited. Cert-manager can even automatically renew them. DEX is installed via Helm Chart; all its settings are contained in a ConfigMap.
Authentication via Keycloak
One of Keycloak's advantages is that it has its own web interface, unlike DEX. It also supports a larger number of backends and can work with more than just Kubernetes. Additionally, Keycloak is ideal for managing user access to multiple applications, as it's designed to work as an SSO (Single Sign-On) server. However, this comes at the cost of a higher learning curve, as even experienced developers unfamiliar with Keycloak will need to study its extensive documentation first.
How Third-Party Authentication Services Work
After the user opens the Gangway form, they will be redirected to the DEX/Keycloak authorization page.
-
The application checks the correctness of the entered data.
-
If the data is correct, DEX/Keycloak returns authentication tokens to Gangway. This process is automated and invisible to the user.
-
The user can then download the generated kubeconfig with access settings based on the received data.
-
Kubeconfig is needed to send requests directly to the Kubernetes server, where DEX/Keycloak checks the validity of the tokens.
Which Method to Choose
To conclude, here are some factors to help you choose between Kubernetes certificates, DEX, and Keycloak:
-
Choose certificate authentication if the project is small and has few users, as tracking them in this method can be inconvenient.
-
Choose DEX if you need access settings only for the cluster, without additional backends.
-
Choose Keycloak if you need to configure access for multiple unrelated applications for individual users.
Kubernetes
19.08.2024
Reading time: 6 min
Share
Similar
Kubernetes
Installing MongoDB in a Kubernetes Cluster
MongoDB is a widely used NoSQL database designed to store large volumes of unstructured data. Combined with Kubernetes, MongoDB becomes a powerful solution for scaling databases efficiently within a unified environment.
Prerequisites
To install MongoDB on Kubernetes, you'll need a configured cloud server (or a physical one) with superuser rights and a Kubernetes cluster. While any OS can be used, Linux is recommended for minimal installation issues.
Step-by-Step MongoDB Installation
Connect to the Server:
Gain superuser access and install necessary software:
sudo -s
apt-get update && apt install curl apt-transport-https -y && curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | apt-key add - && echo "deb https://apt.kubernetes.io/ kubernetes-xenial main" | tee -a /etc/apt/sources.list.d/kubernetes.list && apt-get update && apt install kubectl -y
Configure Kubernetes Environment:
Create a directory, add the configuration file, and set the environment variable:
mkdir /usr/local/etc/mongo && cd /usr/local/etc/mongo
cat << EOF > testcluster.conf<insert your cluster config data here>EOF
echo "export KUBECONFIG=testcluster.conf" >> ~/.bashrc
Verify Connection:
Use kubectl cluster-info to check the connection. A successful connection will display:
Kubernetes control plane is running at <IP>.
Create MongoDB Configuration Files:
Set up a container for data storage and create a Creds.yaml file for MongoDB credentials. Encrypt login and password using BASE64:
echo <unencrypted data> | base64echo <encrypted data> | base64 -d
Example:
apiVersion: v1
data:
username: <username encrypted with BASE64>
password: <password encrypted with BASE64>
kind: Secret
metadata:
creationTimestamp: null
name: creds
Deploy MongoDB:
Create a PersistVolClaim.yaml file with MongoDB configuration and deploy it using:
kubectl apply -f PersistVolClaim.yaml
The file example:
apiVersion: apps/v1
kind: Deployment
metadata:
labels:
app: mongo
name: mongo
spec:
replicas: 1
selector:
matchLabels:
app: mongo
strategy: {}
template:
metadata:
labels:
app: mongo
spec:
containers:
- image: mongo
name: mongo
args: ["--dbpath","/data/db"]
livenessProbe:
exec:
command:
- mongo
- --disableImplicitSessions
- --eval
readinessProbe:
exec:
command:
- mongo
- --disableImplicitSessions
- --eval
env:
- name: MONGO_INITDB_ROOT_USERNAME
valueFrom:
secretKeyRef:
name: creds
key: username
- name: MONGO_INITDB_ROOT_PASSWORD
valueFrom:
secretKeyRef:
name: creds
key: password
volumeMounts:
- name: "datadir"
mountPath: "/data/db"
volumes:
- name: "datadir"
persistentVolumeClaim:
claimName: "mongopvc"
Test MongoDB Connection:
After deploying containers, verify the connection:
kubectl exec deployment/client -it -- /bin/bashmongo
If everything is connected successfully, the system will display a typical database prompt. To create a new database, simply switch to it; however, note that it will not be saved until you add some data. This can be done as follows:
use database_name
db.createCollection("newdata")
show dbs
The last command is used to verify that the newly created database exists.
Considerations for MongoDB in Kubernetes
Remote Storage: For flexibility, use remote storage for MongoDB to facilitate movement if needed.
Resource Management: Configure requests and limits in replica pods to avoid performance issues.
Pod Disruption Budget: Set up to maintain the desired number of running replicas.
Other Tools and Customization
The method of installing MongoDB in Kubernetes described here is one of many options. You can also use software specifically designed to work with Kubernetes, such as Helm or KubeDB. KubeDB, in particular, was created to simplify the integration of other products into Kubernetes. As for Helm, it is another popular solution by VMware (although VMware didn't develop it but acquired and now maintains the product).
Another solution is Percona Operator. This modern, open-source application (developed in 2018) is user-friendly and continuously improved by the community. Some people use combined solutions like Percona + Helm. However, installing MongoDB using each of these applications has its nuances, so it's advisable to study these products before proceeding; plenty of documentation is available.
In conclusion, you can use a customized MongoDB image to manage a MongoDB cluster in Kubernetes according to your specific needs. For example, the default MongoDB image doesn't include authentication. Therefore, you can download an image with pre-configured authentication or create your own. Of course, using customized Docker images is slightly more complex than the implementation described above. Still, it gives you full control over the database configurations and settings according to your requirements. You can find useful information on customizing the official MongoDB image here.
Conclusion
With this guide, you can deploy MongoDB in a Kubernetes cluster. However, further tasks will require some knowledge of Kubernetes, so if you're not familiar with it, we recommend first studying the official documentation.
23 August 2024 · 5 min to read
Kubernetes
Kubernetes Cluster: Installation, Configuration, and Management
Kubernetes, or K8s, is an open-source container orchestration platform developed by Google. The core concept behind Kubernetes is that a user installs it on a server, or more likely a cluster, and deploys various workloads on it. Kubernetes addresses challenges related to container creation, scaling, namespaces, access rights, and more. The primary interaction with the cluster is through YAML configuration files.
This tutorial will guide you through creating and deploying a Kubernetes cluster locally.
Creating Virtual Machines
We will set up the Kubernetes cluster on two virtual machines: one acting as the master node and the other as a worker node. While deploying a cluster with only two nodes is not practical for real-world use, it is sufficient for educational purposes. If you wish to create a Kubernetes cluster with more nodes, simply repeat the process for each additional node.
We will use Oracle's VirtualBox to create virtual machines, which you can download from this link. After installation, proceed to create the virtual machines.
For the operating system, we will use Ubuntu Server, which can be downloaded here. After downloading, open VirtualBox.
Click "Create" in VirtualBox to create a new virtual machine. The default settings are sufficient, but allocate 3 GB of RAM and 2 CPUs for the master node (which manages the Kubernetes cluster) and 2 GB of RAM for the worker node. Kubernetes requires a minimum of 2 CPUs for the master node. Create two virtual machines this way.
After creating the virtual machines, create a boot image with the Ubuntu Server distribution. Go to "Storage" and click "Choose/Create a Disk Image."
Click "Add" and select the Ubuntu Server distribution. Then, start both machines and install the operating system by selecting "Try or Install Ubuntu." During installation, create users for each system and choose the default settings.
After installation, shut down both virtual machines and go to their settings. In the "Network" section, change the connection type to "Bridged Adapter" for each system so that the virtual machines can communicate with each other over the network.
System Preparation
Network Configuration
Set the node names for the cluster. On the master node, execute the following command:
sudo hostnamectl set-hostname master.local
On the worker node, execute:
sudo hostnamectl set-hostname worker.local
If there are multiple worker nodes, assign each a unique name: worker1.local, worker2.local, and so on.
To ensure that nodes are accessible by name, modify the hosts file on each node. Add the following lines:
192.168.43.80 master.local master192.168.43.77 worker.local worker
Here, 192.168.43.80 and 192.168.43.77 are the IP addresses of each node. To find the IP address, use the ip addr command:
ip addr
Locate the IP address next to inet. Open the hosts file and make the necessary edits:
sudo nano /etc/hosts
To verify that the VMs can communicate with each other, ping the nodes:
ping 192.168.43.80
If successful, you will receive a response similar to this:
PING 192.168.43.80 (192.168.43.80) 56(84) bytes of data.64 bytes from 192.168.43.80: icmp_seq=1 ttl=64 time=0.054 ms
Updating Packages and Installing Additional Utilities
Next, install the necessary utilities and packages on each node. These steps should be applied to each node unless specified otherwise. Start by updating the package list and systems:
sudo apt-get update && apt-get upgrade -y
Then install the following packages:
sudo apt-get install curl apt-transport-https git iptables-persistent -y
Swap File
Kubernetes will not start with an active swap file, so it needs to be disabled:
sudo swapoff -a
To prevent it from reactivating after a reboot, modify the fstab file:
sudo nano /etc/fstab
Comment out the line with #:
# /swap.img none swap sw 0 0
Kernel Configuration
Load additional kernel modules:
sudo nano /etc/modules-load.d/k8s.conf
Add the following two lines to k8s.conf:
br_netfilteroverlay
Now, load the modules into the kernel:
sudo modprobe br_netfiltersudo modprobe overlay
Verify the modules are loaded successfully:
sudo lsmod | egrep "br_netfilter|overlay"
You should see output similar to this:
overlay 147456 0br_netfilter 28672 0bridge 299008 1 br_netfilter
Create a configuration file to process traffic through the bridge in netfilter:
sudo nano /etc/sysctl.d/k8s.conf
Add the following two lines:
net.bridge.bridge-nf-call-ip6tables = 1net.bridge.bridge-nf-call-iptables = 1
Apply the settings:
sudo sysctl --system
Docker Installation
Run the following command to install Docker:
sudo apt-get install docker docker.io -y
For more details on installing Docker on Ubuntu, refer to the official guide.
After installation, enable Docker to start on boot and restart the service:
sudo systemctl enable dockersudo systemctl restart docker
Kubernetes Installation
Add the GPG key:
sudo curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo apt-key add -
Next, create a repository configuration file:
sudo nano /etc/apt/sources.list.d/kubernetes.list
Add the following entry:
deb https://apt.kubernetes.io/ kubernetes-xenial main
Update the apt-get package list:
sudo apt-get update
Install the following packages:
sudo apt-get install kubelet kubeadm kubectl
Installation is now complete. Verify the Kubernetes client version:
sudo kubectl version --client
The output should be similar to this:
Client Version: version.Info{Major:"1", Minor:"24", GitVersion:"v1.24.2"}
Cluster Configuration
Master Node
Run the following command for the initial setup and preparation of the master node:
sudo kubeadm init --pod-network-cidr=10.244.0.0/16
The --pod-network-cidr flag specifies the internal subnet address, with 10.244.0.0/16 being the default value.
The process will take a few minutes. Upon completion, you will see the following message:
Then you can join any number of worker nodes by running the following on each as root:kubeadm join 192.168.43.80:6443 --token f7sihu.wmgzwxkvbr8500al \--discovery-token-ca-cert-hash sha256:6746f66b2197ef496192c9e240b31275747734cf74057e04409c33b1ad280321
Save this command to connect the worker nodes to the master node.
Create the KUBECONFIG environment variable:
export KUBECONFIG=/etc/kubernetes/admin.conf
Install the Container Network Interface (CNI):
kubectl apply -f https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml
Worker Node
On the worker node, run the kubeadm join command obtained during the master node setup. After this, on the master node, enter:
sudo kubectl get nodes
The output should be:
NAME STATUS ROLES AGE VERSIONmaster.local Ready control-plane,master 10m v1.24.2worker.local Ready <none> 79s v1.24.2
The cluster is now deployed and ready for operation.
Conclusion
Setting up a Kubernetes cluster involves several steps, from creating and configuring virtual machines to installing and configuring the necessary software components. This tutorial provided a step-by-step guide to deploying a basic Kubernetes cluster on a local environment. While this setup is suitable for educational purposes, real-world deployments typically involve more nodes and more complex configurations. Kubernetes provides powerful tools for managing containerized applications, making it a valuable skill for modern IT professionals. By following this guide, you've taken the first steps in mastering Kubernetes and its ecosystem.
22 August 2024 · 7 min to read
Kubernetes
Running Kubernetes Clusters in the Cloud with VMware
Containerization is an effective way to deliver applications to customers. If your cloud IT infrastructure is deployed on VMware, you can use CSE, or Container Service Extension, to work with Kubernetes (K8s). This solution significantly accelerates the time from receiving code to deploying it in a production cloud system by automating the management (orchestration) of containers with the software.
What is CSE?
CSE is an extension to the VMware vCloud Director (VCD) platform that adds functionality for interacting with Kubernetes clusters—from creation to lifecycle management. Its installation allows for a comprehensive approach, integrating the management of both legacy and containerized applications within a single VMware infrastructure, while maintaining uniformity and a systematic management approach.
Key features
The CSE client facilitates cluster deployment, adds worker nodes, and configures NFS storage.
A vCloud Director-based cloud offers high-security, multi-tenant (user-isolated) computing resources.
The CSE server is a tool for configuring the configuration file and virtual machine templates.
Creating and managing Kubernetes clusters in VMware is relatively complex, especially compared to tools like Docker Swarm, another cluster management tool for remote hosts. Kubernetes is often compared to vSphere, but the discussed platform offers more extensive functionality for managing a containerized IT infrastructure. This compensates for the drawbacks of a complex architecture and the high cost of the product.
CSE Features
The first thing the developers highlight about CSE is the ability to save on the already implemented VMware vCloud Director platform. All previously installed applications will continue to function as before (virtually invisible to the end client), while adding the ability to work with VMware Container. System resilience remains high regardless of traffic uniformity or platform load dynamics.
Benefits of implementing the extension:
A tool for managing clusters, node pools, and other resources.
Significantly reduced time-to-market for any new developments.
Increased availability of web resources, including cloud applications.
Automatic server load distribution.
Improved reliability and performance of CI/CD processes.
The number of containers is unlimited as long as the physical server's resources (memory, CPU, etc.) are sufficient. This allows for parallel development of different projects that are initially isolated from each other. There are also no restrictions on the installed operating systems or programming languages. This is convenient when operating in an international market, even with just one physical server.
Installing the CSE Extension in vcd-cli
The vcd-cli (Command Line Interface) tool manages the infrastructure from the command line. By default, it does not support working with CSE. To enable it, you need to install the container-service-extension add-on:
python3 -m pip install container-service-extension
Next, you need to add the extension to the vcd-cli configuration file, located at ~/.vcd-cli/profiles.yaml. Open this file with a text editor and find the line active with the following value:
extensions:- container_service_extension.client.cse
After saving the changes to the configuration file, log in:
vcd login <host> <organization_name> <login>
Now, verify that the extension is indeed installed and actively interacting with the host:
vcd cse versionCSE, Container Service Extension for VMware vCloud Director, version 3.0.1
vcd cse system infoproperty value----------- ------------------------------------------------------description Container Service Extension for VMware vCloud Directorproduct CSEversion 2.6.1
Creating a Kubernetes Cluster
Next, let's look at activating a Kubernetes cluster within VMware. Integration with the vCloud Director platform allows managing the process from a single point in a familiar interface. Data center resources are typically pooled, and deployment is done through VM templates with pre-installed and pre-configured Kubernetes.
You can create a cluster manually with the command:
vcd cse cluster create <cluster_name> \ --network <network_name> \ --ssh-key ~/.ssh/id_rsa.pub \ --nodes <number_of_nodes> \ --template <template_name>
The cluster and network names are mandatory. The rest are optional and will default if omitted. You can check the full list of active templates with the command:
vcd cse template list
The selected network must be of type Routed and connected to the internet. If either of these conditions is not met, the cluster initialization process will stall during the master node generation. You can use a "grey" network with NAT or Direct Connect technology. The result of the cluster creation will be visible in the vCloud Director platform's web interface, in the vApps section.
After monitoring the status, the final step is to create a configuration file for Kubernetes. Generate it with the command:
vcd cse cluster config <cluster_name> > config
Then move the file to an appropriate location with the commands:
mkdir ~/.kube/configcp config ~/.kube/config
The cluster is now fully ready for use—from setting user parameters to deploying virtual machines, applications, and more. However, keep in mind that emulating containerization does have some limitations.
Implementation Features
For instance, the CSE extension does not support the LoadBalancer service type. Therefore, Kubernetes manifests using it (plus Ingress) will not work correctly. There are solutions to this drawback, and we'll discuss two of the most popular—MetalLB and Project Contour.
MetalLB
Using MetalLB with Kubernetes involves a load balancer that replaces cloud routing protocols with standard LB protocols. Here's an example of how to use it.
1) Create a namespace and add MetalLB using manifests:
kubectl apply -f https://raw.githubusercontent.com/metallb/metallb/v0.9.5/manifests/namespace.yamlkubectl apply -f https://raw.githubusercontent.com/metallb/metallb/v0.9.5/manifests/metallb.yaml
2) Next, configure node connection security. Without this, the transmitted pods will go into a CreateContainerConfigError status, and error messages such as secret memberlist not found will appear in the logs:
kubectl create secret generic -n metallb-system memberlist --from-literal=secretkey="$(openssl rand -base64 128)"
3) Check the current status of the utility. If configured correctly, the controller and speaker will be displayed as running:
kubectl get pod --namespace=metallb-system
NAME READY STATUS RESTARTS AGEcontroller-57f648cb96-2lzm4 1/1 Running 0 5h52mspeaker-ccstt 1/1 Running 0 5h52mspeaker-kbkps 1/1 Running 0 5h52mspeaker-sqfqz 1/1 Running 0 5h52m
4) Finally, manually create a configuration file:
apiVersion: v1 kind: ConfigMap metadata: namespace: metallb-system name: config data: config: | address-pools: - name: default protocol: layer2 addresses: - X.X.X.101-X.X.X.102
You should fill in the addresses parameter with the addresses that remain free and will handle the load balancing. Apply the configuration file:
kubectl apply -f metallb-config.yaml
The procedure for setting up a LoadBalancer for Kubernetes using MetalLB is complete; next is Ingress support, which is easier to implement with another tool.
Project Contour
Create a manifest with Project Contour using the command:
kubectl apply -f https://projectcontour.io/quickstart/contour.yaml
This command automatically deploys the Envoy proxy server, which listens on the standard ports 80 and 443.
Conclusion
Integrating Kubernetes into VMware with the Container Service Extension (CSE) unifies the management of legacy and containerized applications within VMware vCloud Director. While the setup may be complex, CSE enhances application deployment, scaling, and management, offering a resilient and scalable infrastructure. Despite some limitations, such as native LoadBalancer support, tools like MetalLB and Project Contour provide effective solutions. Overall, CSE empowers organizations to modernize their IT infrastructure, accelerating development and optimizing resources within a secure, multi-tenant cloud environment.
22 August 2024 · 7 min to read
Do you have questions,
comments, or concerns?
Our professionals are available to assist you at any moment,
whether you need help or are just unsure of where to start.
whether you need help or are just unsure of where to start.
