Terraform code organization best practices

Terraform is the most popular coding language for setting up infrastructure out there. It’s become so popular because it’s far better than the alternatives around. Better means easier to write, easier to learn, and most importantly easier to maintain. HCL (HashiCorp Configuration Language) plays a core role in the ease of writing and the maintainability when compared to its competitors.

I’ve been using Terraform for more than 6 years if not daily, weekly for sure, creating solutions for some of the biggest companies out there across different industries. And I’ll share two important topics I gathered during that work:

1. Common pitfalls that will give you headaches in the long run if you don’t start right.
2. Best practices on how to organize your code inside repositories to make sure it’s as easy as possible to maintain it.

Common pitfalls

All these pitfalls are totally understandable if you’re a beginner in Terraform, or if you have a huge pressure on your shoulders to deliver more infrastructure. But once you know them, it’s not an excuse anymore. The rework you’ll have in the long run if you fall for these pitfalls is enormous.

1. One big state file (or many big state files)

Extending the concept of using small files learnt with the Clean Code, a big state file will happen if you have too many responsibilities in your configuration (I’ll approach what is a Terraform configuration on the best practices below). So if you need to provision an S3 bucket, an EFS volume and a couple EC2 instances be sure to end up with at least 3 state files (3 per environment). Depending on how complex your EC2 module will get, you might want to consider breaking it into two. As an example: if you’re creating just the ENI, the EC2, and a couple EBS, that’s fine. But if you have a complex userdata to build, that’s another module and configuration, and therefore another state file. Also, if you have multiple EBS volumes to attach to a complex solution like SAP, that demands a new configuration/state file.

Now why is a big state file bad? Simply put, a big state file will always drag tons of dependencies at every Terraform plan. For example, you might want to do a very simple update in the lifecycle policy for your EFS files. But if your state file also includes your RDS, sooner or later you’ll see something changing and Terraform willing to destroy your database. So, be cautious.

2. All the code everywhere all at once

The second most common mistake is duplicating code (as it always is). Duplicate code will give you headaches when you have to do a change that should affect all your S3 buckets, for example. Let’s say you were just given a task to update the S3 bucket policies to allow an auditing account to read your files. If you don’t have a common single module that handles your buckets, you might very well consider the task done after changing 3 or 4 of them, and you’ll miss a couple more because it’s somewhere else in the code. That will generate rework in your team, and reduce your team’s trust from other teams.

3. Not merging your branches

This is straightforward. You have one branch per environment: dev, QA, prod. If you’re not testing your code in dev, then merging to QA, deploying, testing and only then merging to prod, you’re not trusting your code. It means you’re making changes straight to the environment branch without testing it in a lower environment before. So keep the code flowing from lower to higher branches. If you can’t merge it, red flag. The code should have variables that make it behave differently in different environments, so your merging keeps flowing.

How to organize your code

I’m suggesting three ways of organizing your code from worst to best. Head straight to the last one if you have a big team (~10 folks or more) working on provisioning your infra across multiple accounts. First and second suggestions are here just to cover some scenarios where companies have to adhere to policies like SOX, SCI, etc.

1. One repo: configurations and modules in just one repository.

In this case you have one repository. Pretty easy. Inside that, split your code into two folders: configurations and modules. This scenario will work pretty well if you have a small team (3 folks max) handling infrastructure for a small number of apps (max 5).

Modules: one folder per mode. You can consider modules as functions you’d write in an imperative language. You write modules in a generic way so you can reuse them as many times as you want.

Configurations: one folder per configuration. Every time you want to deploy one of your modules, you call them in a configuration. So the configuration is where you run your Terraform init/plan/apply from.

Example of how it’s gonna be in the end:

cloud-infra:

• folder — modules:
• EFS
• S3
• EC2
• Lambda
• IAM
• EKS
• ECR
• folder — configurations:
• EFS
• S3
• EC2
• Lambda
• Etc

2. One repo for modules and X repos for the configurations

In this case you’ll have one repository with the same contents of the folder modules above. What changes is that it gives you more flexibility and isolation for handling multiple applications. Imagine you have microservices, or multiple apps to deploy like Sonarqube, Jira, Prometheus, etc. In this case you will have one repository per application you need to deploy consuming the modules from a common repository. So when you’re changing Sonarqube’s database you’re sure you’re not touching Jira’s DB or Prometheus logs. And if you’re using microservices, you’re changing just the infra related to that specific microservice.

Example of how it’s gonna be in the end:

terraform-modules:

• EFS
• S3
• EC2
• IAM
• Lambda
• Etc

sonarqube:

• app folder 1
• app folder 2
• config folder
• terraform

microservice-1:

• app folder 1
• app folder 2
• config folder
• terraform

cloud-network:

• Terraform code in the root

cloud-security:

• Terraform code in the root

3. One repo per module and one repo per configuration

This is the most scalable and easiest to maintain in the long run. It also requires maturity and ownership from the team maintaining it.

Example of how it’s gonna be in the end:

TF-module-EFS:

• Terraform code in the root

TF-module-S3:

• Terraform code in the root

sonarqube:

• app folder 1
• app folder 2
• config folder
• terraform

microservice-1:

• app folder 1
• app folder 2
• config folder
• terraform

cloud-network:

• Terraform code in the root

cloud-security:

• Terraform code in the root

I hope it helps with avoiding issues while maintaining your Terraform infrastructure in the long run. If you’ve seen other patterns or see any flaws in the ones above, please share in the comments!