How to Deploy Jenkins CI/CD Pipelines with Terraform

Modern software development requires fast and high-quality delivery of new features and fixes. CI/CD (Continuous Integration and Delivery) automation allows you to:

• Reduce time-to-market: Developers can quickly validate code and deliver it to the production environment.
• Improve code quality: Automated testing detects errors at early stages, reducing the cost of fixing them.
• Increase reliability and stability: CI/CD pipelines minimize the human factor, ensuring process consistency.
• Ensure flexibility and scalability: It becomes easier to adapt to changing requirements and increase workloads.

Benefits of Using Jenkins and Terraform Together

Jenkins and Terraform are a powerful combination of tools for CI/CD automation:

Jenkins:

• Provides flexible pipeline configuration options.
• Integrates with many tools and version control systems (Git, GitHub, Bitbucket).
• Supports pipeline customization through Jenkinsfile.

Terraform:

• Simplifies infrastructure creation and management through declarative code.
• Supports multi-cloud environments, making it a universal deployment solution.
• Allows easy tracking of infrastructure changes through its state management system.

By using Jenkins and Terraform together, you can:

• Automate infrastructure deployment: Terraform creates the environment for running the application.
• Optimize CI/CD pipelines: Jenkins handles build, test, and deploy using the resources provisioned by Terraform.
• Reduce time and effort in infrastructure management: The entire process becomes manageable through code (IaC).

Deployment

Registration and Initial Setup in Hostman

1. Go to the Hostman website and create an account.
2. Create a new project to separate CI/CD resources from other projects.
3. Configure access keys. In the API section, create an API key for working with Terraform. Save the key in a secure place, as it will be needed for configuration.

Installing and Configuring Terraform

1. Download Terraform from the HashiCorp website.
2. Install it on your system:
   • For Windows: add the path to the Terraform binary to environment variables.
   • For macOS/Linux: move the terraform file to /usr/local/bin/.
3. Verify installation:

terraform --version

Creating Hostman Infrastructure

Next, we will use Terraform to create the necessary infrastructure. We recommend using at least 2 CPUs, 4 GB RAM for a basic Jenkins setup, and 15–40 GB disk for temporary files and artifacts.

Create the configuration file provider.tf:

terraform {
  required_providers {
    hm = {
      source = "hostman-cloud/hostman"
    }
  }
  required_version = ">= 0.13"
}

provider "hm" {
  token = "your_API_key"
}

Describe infrastructure for Jenkins deployment in main.tf:

data "hm_configurator" "example_configurator" {
  location = "us-2"
}

data "hm_os" "example_os" {
  name    = "ubuntu"
  version = "22.04"
}

resource "hm_ssh_key" "jenkins_key" {
  name = "jenkins-ssh-key"
  body = file("~/.ssh/id_rsa.pub")
}

resource "hm_vpc" "jenkins_vpc" {
  name        = "jenkins-vpc"
  description = "VPC for Jenkins infrastructure"
  subnet_v4 = "192.168.0.0/24"
  location = "us-2"
}

resource "hm_server" "jenkins_server" {
  name     = "Jenkins-Server"
  os_id    = data.hm_os.example_os.id
  ssh_keys_ids = [hm_ssh_key.jenkins_key.id]

  configuration {
    configurator_id = data.hm_configurator.example_configurator.id
    disk = 15360 
    cpu  = 2     
    ram  = 4096  
  }

  local_network {
    id = hm_vpc.jenkins_vpc.id
    ip = "192.168.0.10" # Static IP within the VPC subnet
    mode = "dnat_and_snat"
  }

  connection {
    type        = "ssh"
    user        = "root"
    private_key = file("~/.ssh/id_rsa")
    host        = self.networks[0].ips[0].ip # Correct way to get IP
    timeout     = "10m"
  }

  provisioner "remote-exec" {
    inline = [
      "apt-get update -y",
      "apt-get install -y ca-certificates software-properties-common",
      "add-apt-repository -y ppa:openjdk-r/ppa",
      "apt-get update -y",
      "apt-get install -y openjdk-17-jdk",
      "curl -fsSL https://pkg.jenkins.io/debian-stable/jenkins.io-2023.key | tee /usr/share/keyrings/jenkins-keyring.asc > /dev/null",
      "echo deb [signed-by=/usr/share/keyrings/jenkins-keyring.asc] https://pkg.jenkins.io/debian-stable binary/ | tee /etc/apt/sources.list.d/jenkins.list > /dev/null",
      "apt-get update -y",
      "apt-get install -y jenkins",
      "update-alternatives --set java /usr/lib/jvm/java-17-openjdk-amd64/bin/java",
      "sed -i 's|JAVA_HOME=.*|JAVA_HOME=/usr/lib/jvm/java-17-openjdk-amd64|' /etc/default/jenkins",
      "systemctl daemon-reload",
      "systemctl start jenkins",
      "ufw allow 8080"
    ]
  }
}

output "jenkins_url" {
  value = "http://${hm_server.jenkins_server.networks[0].ips[0].ip}:8080"
}

Install the Hostman Terraform Provider:

terraform init

Check the connection to Hostman:

terraform plan

After successful initialization, you should see the following message:

Apply the changes:

terraform apply

Confirm execution by typing yes.

When you go to your Hostman project, you will see a new server being created. Take note of the resources—they match exactly what we specified in the main.tf configuration.

After the server is created, go to the IP address with port 8080, where Jenkins should now be running. This address is also visible in the control panel, on the server Dashboard.

You will see Jenkins’ initial setup screen:

Configuration Breakdown

In this section, we will go through each part of the code in detail.

Data Sources (Fetching Infrastructure Information)

Fetching the configurator ID for the selected region. Needed to define available hardware configurations (CPU/RAM/Disk):

data "hm_configurator" "example_configurator" {
  location = "us-2"
}

Fetching the OS image ID for Ubuntu 22.04.

data "hm_os" "example_os" {
  name    = "ubuntu"
  version = "22.04"
}

Resources (Creating Infrastructure)

Creating an SSH key for server access:

resource "hm_ssh_key" "jenkins_key" {
  name = "jenkins-ssh-key"
  body = file("~/.ssh/id_rsa.pub")
}

Creating a private network (VPC) for Jenkins isolation:

resource "hm_vpc" "jenkins_vpc" {
  name        = "jenkins-vpc"
  description = "VPC for Jenkins infrastructure"
  subnet_v4 = "192.168.0.0/24"
  location = "us-2"
}

Server Configuration

Main server parameters:

• name: Server name
• os_id: OS image ID
• ssh_keys_ids: SSH key binding

resource "hm_server" "jenkins_server" {
  name     = "Jenkins-Server"
  os_id    = data.hm_os.example_os.id
  ssh_keys_ids = [hm_ssh_key.jenkins_key.id]

Server characteristics:

• configurator_id: Link to configurator
• disk: Disk size (15 GB)
• cpu: Number of cores
• ram: Memory size (4 GB)

configuration {
  configurator_id = data.hm_configurator.example_configurator.id
  disk = 15360 
  cpu  = 2     
  ram  = 4096  
}

Private network settings:

• id: ID of created VPC
• ip: Static IP within subnet
• mode: NAT mode

local_network {
  id = hm_vpc.jenkins_vpc.id
  ip = "192.168.0.10"
  mode = "dnat_and_snat"
}

Connection Settings

SSH connection parameters for provisioning:

connection {
  type        = "ssh"
  user        = "root"
  private_key = file("~/.ssh/id_rsa")
  host        = self.networks[0].ips[0].ip
  timeout     = "10m"
}

Provisioning (Software Installation)

provisioner "remote-exec" {
  inline = [
    # Package update
    "apt-get update -y",
    
    # Installing dependencies
    "apt-get install -y ca-certificates software-properties-common",
    
    # Adding Java repository
    "add-apt-repository -y ppa:openjdk-r/ppa",
    
    # Installing Java 17
    "apt-get install -y openjdk-17-jdk",
    
    # Adding Jenkins repository
    "curl -fsSL ...",
    "echo deb ...",
    
    # Installing Jenkins
    "apt-get install -y jenkins",
    
    # Java configuration
    "update-alternatives --set java ...",
    "sed -i 's|JAVA_HOME=.*|...|' /etc/default/jenkins",
    
    # Starting service
    "systemctl daemon-reload",
    "systemctl start jenkins",
    
    # Opening port
    "ufw allow 8080"
  ]
}

Output Information

Displaying the URL for accessing Jenkins:

output "jenkins_url" {
  value = "http://${hm_server.jenkins_server.networks[0].ips[0].ip}:8080"
}

Potential Issues with Jenkins and Terraform

Secret and Confidential Data Management

Problem: Storing API keys, SSH keys, and other secrets may be unsafe if added as plain text.

Solution: Use secret managers such as HashiCorp Vault, or Jenkins integration with credential management systems. In Terraform, specify secrets via environment variables or remote storage.

Terraform State Management Issues

Problem: The state file (terraform.tfstate) may be corrupted or overwritten if multiple users access it simultaneously.

Solution: Configure a remote backend (e.g., object storage) to store the state. Enable state locking to prevent simultaneous command execution.

Complexity of Terraform and Jenkins Integration

Problem: Integration can be difficult for beginners due to differences in configuration and infrastructure management.

Solution: Use the Terraform plugin for Jenkins to simplify setup. Create detailed documentation and use Jenkinsfile templates for repeatable processes.

Jenkins and Terraform Updates

Problem: New versions of Jenkins and Terraform may be incompatible with existing configurations.

Solution: Test updates in local or test environments. Regularly update Jenkins plugins to maintain compatibility.

Hostman Limitations and Workarounds

Hostman API Limitations

Problem: Not all actions or resources may be available through the API.

Solution: Use a combination of Terraform and the Hostman interface. Leave operations unavailable via API for manual execution or scripts in Python/Go.

Resource Limitations

Problem: Limits on the number of servers, storage, or networking resources may cause scaling delays.

Solution: Optimize existing resource usage. Plan workload ahead, increasing limits through Hostman support.

Network Infrastructure Scalability

Problem: Deploying large applications may be difficult due to complex network configurations (e.g., VPC).

Solution: Break infrastructure into Terraform modules. Use pre-prepared network architecture templates.

Lack of Deep Integrations

Problem: Hostman may not provide full integrations with external services, such as GitHub Actions or third-party monitoring.

Solution: Configure webhook notifications for integration with Jenkins or other CI/CD systems. Use external APIs of third-party services for custom integrations.

Debugging Recommendations

Terraform Debugging

• Use the terraform plan option to validate configuration before applying changes.

For detailed analysis, add the -debug flag:

terraform apply -debug

• Verify provider settings (e.g., token and API keys).

Jenkins Pipeline Debugging

• Enable logging in Jenkins: specify --verbose or --debug in Terraform and Docker build commands.
• Configure artifact archiving in Jenkins to save error logs.
• Split steps in Jenkinsfile to isolate problematic stages.

Monitoring and Notifications

• Install monitoring plugins in Jenkins (e.g., Slack or Email Extension) for failure notifications.
• Use external monitoring (e.g., Zabbix or Prometheus) to monitor server status in Hostman.

Backups of Terraform State and Jenkins Data

• Configure regular backups of the state file (terraform.tfstate) to remote storage.
• Create backups of Jenkins configuration and job data.

Troubleshooting Common Errors

• Jenkins server connection error: Check SSH keys and firewall rules.
• Terraform resource creation error: Verify API key validity and available quotas.
• Pipeline hang: Ensure Docker containers are correctly downloaded and running.

Efficient work with Jenkins, Terraform, and Hostman requires awareness of possible issues and systematic resolution. Use the recommendations above to prevent errors, optimize resources, and simplify debugging of your CI/CD processes.

Recommendations for Beginners and Professionals

For Beginners:

• Start simple: Learn basic Terraform commands such as init, plan, apply. Configure Jenkins using pre-built images and plugins.
• Use documentation and templates: Explore Hostman API docs and Terraform module examples. Use Jenkinsfile templates to learn pipeline writing faster.
• Test locally: Run a local Jenkins instance and test small Terraform configurations before moving to the cloud.
• Experiment: Start with small projects to understand CI/CD and infrastructure management principles. Reach out to the Hostman community or support if issues arise.

For Professionals:

• Optimize processes: Create custom Terraform modules for reuse across projects. Configure Jenkins for parallel builds to reduce pipeline execution time.
• Focus on security: Configure secure storage of keys and sensitive data using Vault or Jenkins built-in tools. Protect terraform.tfstate by using remote storage with HTTPS access.
• Develop multi-cloud approach: Integrate Hostman with other platforms to create backup pipelines for critical applications. Use Terraform to manage cross-cloud infrastructure.
• Adopt new tools: Explore containerization and orchestration with Kubernetes and Docker. Integrate Jenkins with monitoring systems such as Prometheus or Grafana for performance analysis.

Jenkins and Terraform combined with Hostman open wide opportunities for developers and DevOps engineers. Beginners can easily learn these tools by starting with simple projects, while professionals can implement complex CI/CD pipelines with multi-cloud infrastructure. This approach not only accelerates development but also helps create a scalable, secure, and fault-tolerant environment for modern applications.

Resources for Learning

Terraform Documentation:

• Terraform Official Website
• Terraform Module Registry

Jenkins Guides:

• Official Jenkins Documentation
• Jenkins Plugins
• Article: How to Automate Jenkins Setup with Docker

Hostman:

• Hostman API Documentation
• Hostman Terraform Provider Documentation
• Hostman user guides

Useful Tools:

• VS Code for working with Terraform and Jenkins
• HashiCorp Vault for secure token storage

Communities:

• DevOps on Reddit
• Stack Overflow: Terraform