Terraform for DevOps: Best Practices

Terraform is one of the most popular Infrastructure as Code (IaC) tools that allows you to manage infrastructure using various cloud providers. It follows a declarative approach, meaning you describe the desired infrastructure rather than writing step-by-step instructions for its creation—Terraform then automatically provisions it.

This tutorial outlines best practices for efficient Terraform development.

File and Folder Structure

Maintaining a clear and organized file structure is crucial when working on a large project with complex infrastructure. A consistent folder and file structure improves project maintainability.

A project example:

-- PROJECT-DIRECTORY/
-- modules/
      -- <service1-name>/
         -- main.tf
         -- variables.tf
         -- outputs.tf
         -- provider.tf
         -- README
      -- <service2-name>/
         -- main.tf
         -- variables.tf
         -- outputs.tf
         -- provider.tf
         -- README
      -- ...other…
 -- environments/
      -- dev/
         -- backend.tf
         -- main.tf
         -- outputs.tf
         -- variables.tf
         -- terraform.tfvars
 -- qa/
         -- backend.tf
         -- main.tf
         -- outputs.tf
         -- variables.tf
         -- terraform.tfvars
-- stage/
         -- backend.tf
         -- main.tf
         -- outputs.tf
         -- variables.tf
         -- terraform.tfvars
-- prod/
         -- backend.tf
         -- main.tf
         -- outputs.tf
         -- variables.tf
         -- terraform.tfvars

Separating Configuration Files

Keeping all the code in a single main.tf file is a bad idea.  Instead, split it into separate files based on their purpose:

• main.tf – Calls modules and data sources to create resources.

• variables.tf – Defines variables used in main.tf.

• outputs.tf – Specifies the outputs of resources created in main.tf.

• versions.tf – Defines Terraform and provider version requirements.

• terraform.tfvars – Contains variable values.

Module Structure

The standard module structure is described in the Terraform documentation.

Follow these best practices:

• Group resources based on their purpose, e.g., vps.tf, s3.tf, load_balancer.tf.

• Avoid creating a separate file for each resource unless necessary.

• Include a README.md file for each module with a clear description of its purpose.

Directory Separation for Applications and Environments

• To independently manage infrastructure for different applications, place resources for each application in its own directory.

• Store shared resources (e.g., networks) in a dedicated common resources directory.

• Use separate directories for each environment (dev, qa, stage, production).

• Use modules to share common code across all environments.

• Each environment directory should contain:

• • backend.tf – Defines the Terraform backend state configuration.

  • main.tf – Contains the infrastructure description.

Static Files and Templates

• Static files should be stored in the files/ directory.

• Template files should be placed in the templates/ directory.

General Code Structuring Guidelines

• Place count or for_each as the first argument within a resource or data source block, followed by a new line. Always list tags, depends_on, and lifecycle as the last arguments in a consistent order, separated by a single blank line.

• Keep resource modules simple—avoid unnecessary complexity.

• Avoid hardcoding values—use variables or data sources instead.

Naming, Code Style, and File Formatting

Terraform code should be written in a way that other developers can easily understand. Consistent naming conventions improve code readability and maintainability.

Key Naming Rules

• Use underscore (_) instead of hyphen (-) to separate multiple words in names.

• Use only lowercase letters and numbers.

• Use singular nouns for resource names.

• Do not repeat the resource type in its name:

• Good example: resource "aws_route_table" "public" {}

• Bad example: resource "aws_route_table" "public_route_table" {}

• Differentiate resources of the same type using descriptive names (e.g., primary, secondary, public, private).

Variable Management Rules

• Declare all variables in the variables.tf file.

• Use descriptive names that reflect the variable's purpose.

• Provide meaningful descriptions for all variables—this helps generate module documentation and provides context for new developers.

• Organize variable keys in the following order:

• description

• type

• default

• validation

• Provide default values whenever possible:

• Specify a default if a variable has environment-independent values (e.g., disk size).

• If a variable has environment-specific values, do not set a default.

• Use plural names for variables of type list(...) or map(...).

• Prefer simple types (number, string, list(...), map(...), any) over object(), unless strict key constraints are needed.

• For boolean variables, use positive names (e.g., enable_external_access).

• For numeric input/local/output variables, include measurement units in the name (e.g., ram_size_gb).

• Use binary prefixes for storage units (kilo, mega, giga).

Output Value Rules

• Organize all outputs in the outputs.tf file.

• Output all useful values that other modules may need.

• Provide meaningful descriptions for all output values.

• Name outputs based on their contents, following this structure: {name}_{type}_{attribute}

• Document outputs in README.md.

• Use tools like terraform-docs to auto-generate documentation when committing code.

Using Built-in Formatting Tools

• Use the terraform fmt command to format Terraform files according to its canonical style.

• All Terraform files must comply with terraform fmt standards to ensure consistency.

Best Security Practices in Terraform

Terraform interacts with cloud infrastructure using sensitive data such as API keys. To protect your infrastructure, follow these security best practices:

Secure the Terraform State File

• Never store the state file locally or in version control (e.g., Git). Instead, use Terraform Remote State (e.g., Hostman S3 Storage, AWS S3, Azure Storage). The state file contains sensitive values in plain text, posing a security risk.

• Add Terraform state files to .gitignore to prevent accidental commits.

• Encrypt state files as an extra security measure.

• Regularly back up state files in case of corruption or accidental loss.

• Use one state file per environment (e.g., dev, staging, production).

Enable State Locking

Multiple developers running Terraform commands simultaneously can lead to state corruption and data loss.

To prevent conflicts, enable state locking, which ensures only one user modifies the state at a time.

Note that not all backends support built-in locking. Azure Blob Storage supports locking natively, while AWS S3 supports locking when used with DynamoDB.

Keep Secrets Out of the State File

Terraform stores secrets in plain text within the state file. Avoid storing secrets directly in Terraform configuration.

Instead, use secret management tools (AWS Secrets Manager, HashiCorp Vault, Azure Key Vault) and reference secrets using data sources rather than hardcoding them.

Minimize the Blast Radius

The blast radius refers to the potential impact of failure in your infrastructure. To reduce risk:

• Deploy smaller, isolated infrastructure components.

• Manage critical resources separately from less essential ones.

• Implement least privilege access for Terraform execution roles.

Perform Security Audits

Run security checks after every terraform apply. Use security auditing tools like InSpec or Serverspec. These tools ensure that infrastructure remains in a secure state.

Use the sensitive Flag for Variables

Terraform configurations often contain sensitive inputs like passwords, API tokens, and private keys.

To prevent accidental exposure, mark variables as sensitive:

variable "db_password" {
  description = "Database admin password"
  type        = string
  sensitive   = true
}

Terraform will hide these values in the console and logs.

However, sensitive does not encrypt the data—other precautions are still necessary.

Use .tfvars Files for Variable Definitions

Instead of defining many variables manually, store them in .tfvars files:

db_password = "super_secret_password"

Pass the file during execution:

terraform apply -var-file="secrets.tfvars"

Terraform automatically loads .tfvars files if they exist.

Always store secrets locally using -var-file rather than committing them to version control.

Using Modules in Terraform

Modules are designed for code reuse and help organize infrastructure as code effectively.

Use Common Modules

Use official Terraform modules whenever possible. There is no need to reinvent a module that already exists.

Each module should focus on a single aspect of infrastructure, such as creating database instances.

Tag Module Versions

Sometimes, breaking changes are required in modules. Use version tags so that users can lock their configurations to a specific version and avoid unexpected issues.

Avoid Declaring Providers and Remote State in Shared Modules

Shared modules should not declare providers or remote state. Instead, define providers and remote state configuration in the root modules.

Expose Outputs for All Resources

Variables and outputs help define dependencies between modules and resources. Users cannot properly integrate your module into their Terraform configurations without output values.

Include at least one output that references each resource in a shared module.

Use Submodules for Complex Logic

Submodules help break down complex Terraform logic into smaller, reusable units. This approach reduces code duplication for shared resources.

• Store submodules in modules/$modulename.

• Consider modules as private unless the module documentation states otherwise.

Minimize Resources in Each Root Module

Avoid large root configurations that contain too many resources in a single directory and state file.

Smaller modules make infrastructure easier to manage and faster to deploy.

Additional Recommendations

Version Control

• Store infrastructure code in version control just like application code to maintain history and allow easy rollbacks.

• Follow a branching strategy such as GitFlow.

• Use separate branches for environment-specific root configurations, if necessary.

Best Practices for Testing

Static Analysis

• Validate configuration syntax and structure before deploying resources using linters and dry-run tools.

• Use terraform validate and tools like tflint, config-lint, Checkov, Terrascan, tfsec, Deepsource.

Integration Testing

• Test modules in isolation to ensure correctness.

• Use testing frameworks like Terratest, Kitchen-Terraform, InSpec

Plan Before Applying

• Always check the output of terraform validate and terraform plan before applying changes to an environment.

Keep Terraform Up to Date

Stay on the latest Terraform version whenever a major release occurs.

Run terraform -v to check for updates.

Protect Stateful Resources

Enable deletion protection for stateful resources like databases to prevent accidental removal.

Use the self Variable

The self variable helps when values are unknown before deployment.

Example: If you need the IP address of an instance, but it’s only available after deployment, you can use self to reference it dynamically.

Use Terraform Workspaces

Workspaces allow managing multiple instances of the same Terraform configuration, each with its own state.

Useful for handling multiple environments (e.g., dev, staging, production) using the same Terraform codebase.

For example, here is how you could use workspaces to manage the dev and prod environments:

terraform workspace new dev
terraform apply

terraform workspace new prod
terraform apply