Terraform Style Guide
This is Gruntwork’s style guide for Terraform. It aims to help us ensure that the code we write is clear, readable, idiomatic Terraform code. The conventions detailed in this guide are our preferences and should be thought of as guidelines rather than hard rules.
Starting point
We follow the official HashiCorp style guide for Terraform.
All of these are enforced using terraform fmt
. All Gruntwork Terraform repos should enforce this using pre-commit
hooks; please add if missing.
On top of the official guide, Gruntwork follows some additional conventions.
Additional conventions
General
2 space indentations
Block contents should be indented with 2 spaces.
120 column limit
We follow a 120 column line-length limit, except for description strings in variable
and output
blocks, where single
line strings are preferred.
Block Labels, Variables, and Outputs should be snake case
The label for blocks should be in snake case. E.g. example_instance
, not ExampleInstance
or example-instance
.
Labels are the strings that follow block names. For example, in the following, aws_instance
and example_instance
are labels for the resource
block.
resource "aws_instance" "example_instance" {
# Omitted for brevity
}
This includes variables and outputs as well:
variable "vpc_id" {}
output "instance_name" {}
Module folder conventions
Each module repo should have the following 3 folders:
-
modules
: Terraform modules that are designed to be consumed by users. The intention is that users should pull the modules in themodules
folder in their terraform code usingmodule
blocks. -
examples
: Folder that contains top level Terraform modules that provide an example of how to use the modules in themodules
folder. Theexamples
folder often has subfoldersfor-learning-and-testing
andfor-production
that contain corresponding example code. See Testing: Terratest for more info on how these examples should be organized. -
test
: Terratest Go files for testing the code in the repo. See Testing: Terratest for specific conventions around Terratest.
Additionally, each module in modules
should be organized with the following files:
-
variables.tf
: Allvariable
blocks should go in here and they specify the inputs. -
outputs.tf
: Alloutput
blocks should go in here and they specify the outputs. -
main.tf
: All other logic should be added here. -
dependencies.tf
(optional): Any external references that are pulled in by adata
source block should go in here. This allows consumers of the module to quickly scan for what resources need to already exist to deploy the module.
Any nonstandard file structure should be called out in the README (e.g., if main.tf
is split up into multiple smaller
terraform files).
variables.tf
conventions
Each variable block should always define a description
and type
, even if it is of the string
type (the default), in that order. E.g.:
variable "example" {
description = "This is an example"
type = string
default = "example" # NOTE: this is optional
}
Complex types
Prefer concrete objects (object type) over
free form maps. However, for particularly large objects it is useful to support optional attributes. This is currently
not supported in terraform, so workaround by using any
type.
When using any
type, always use comments to describe the supported attributes.
Example.
outputs.tf
conventions
Each output block should always define a description
, before the value
:
output "greeting" {
description = "This is a greeting for everyone."
value = "hello world!"
}
main.tf
conventions
main.tf
should (loosely) be organized by sections that correspond to components. There is no standard on grouping, but
as a rule of thumb each section should be focused on a specific component of the module. For example, an ECS service
module may consist of the following sections:
- The ECS service resource, and any locals logic for setting up the attributes of the resource.
- The ECS task definition resource, and any locals and template logic for setting up the attributes of the resource (e.g. the container definition).
- Any resources related to configuring ELBs to forward traffic to the ECS service (e.g., listeners and target groups).
- Any resources related to configuring IAM permissions for the ECS service.
- Any resources related to configuring network access (e.g., security group rules) for the ECS service.
There is no standard on ordering the sections, but as a rule of thumb the following sections should be placed first, in order:
- Version constraints for the module
- Provider blocks, if needed.
- The main component of the module (e.g., the
aws_ecs_service
resource for the ECS service module). - All other sections.
- Any
data
blocks (at the bottom).
Conditionals
Use ()
to break up conditionals across multiple lines.
Examples:
locals {
elb_id = (
var.elb_already_exists
? var.elb_id
: module.elb.elb_id
)
excluded_child_account_ids = (
var.config_create_account_rules
? []
: [
for account_name, account in module.organization.child_accounts
: account.id if lookup(lookup(var.child_accounts, account_name, {}), "enable_config_rules", false) == false
]
)
}
Comments
This section lists the Gruntwork conventions around comments in Terraform code.
Prefer #
over //
Use #
for comment strings, not //
or /**/
.
Prefer # -
over # ~
Delimit section header comment blocks with # ----
instead of # \~~~~
.
variables.tf
variables.tf
files should clearly indicate required environment variables, and separate out required variables from
optional variables (with defaults) using block comments.
Example:
## ---------------------------------------------------------------------------------------------------------------------
## ENVIRONMENT VARIABLES
## Define these secrets as environment variables
## ---------------------------------------------------------------------------------------------------------------------
## TF_VAR_master_password
## ---------------------------------------------------------------------------------------------------------------------
## MODULE PARAMETERS
## These variables are expected to be passed in by the operator
## ---------------------------------------------------------------------------------------------------------------------
variable "required_var" {
description = "This variable must be set in order to create the resource."
type = string
}
## ---------------------------------------------------------------------------------------------------------------------
## OPTIONAL PARAMETERS
## These variables have defaults and may be overridden
## ---------------------------------------------------------------------------------------------------------------------
variable "optional_var" {
description = "This variable has a sensible default so it is not necessary to set it explicitly for this module to work."
type = string
default = "Hello world"
}
main.tf
Section comments
Each section (as described in conventions) of main.tf
should have block comments describing the component
managed in the section.
Example:
## ---------------------------------------------------------------------------------------------------------------------
## ONE LINE SUMMARY DESCRIBING WHAT IS BEING MANAGED IN THIS SECTION IN ALL CAPS
## The rest of the comments should be in standard casing. This section should contain an overall description of the
## component that is being managed, and highlight any unconventional workarounds or configurations that are in place.
## ---------------------------------------------------------------------------------------------------------------------
Testing: Terratest
Gruntwork uses Terratest to write tests for Terraform modules. Terratest is a Go library that provides patterns and helper functions for testing infrastructure code.
Terratest best practices
Follow all the best practices listed in Terratest best practices.
The rest of the items below are additional conventions on top of the documented best practices that Gruntwork follows when writing tests using Terratest for terraform modules.
Code formatting
Terratest is a Go library, so each test will be written in Go. All Go source files should be formatted using goimports
and go fmt
.
examples
and tests
In many cases the individual modules in the modules
folder are narrowly focused to a specific subset of the overall
infrastructure. This means that in many cases you will need to provide dependent resources externally to the module in
order to actually deploy them. The Terraform modules in the examples
folder serves this purpose, specifying test
resources that are injected as dependencies to the modules.
As such, the tests should be written against the examples
folder, as opposed to the modules
folder directly. In
other words:
- Every module in
modules
should have a corresponding example module inexamples
that calls it. (NOTE: you can have a single example call multiple modules). - Every example should have at least one test that calls it.
- Tests should not directly call modules in the
modules
folder. Always go through theexamples
.
Parallel
Every test should have the t.Parallel
call in the test function unless there is a specific need to run tests serially,
e.g. manipulating process global resources, like environment variables. This is so that tests run as quickly as possible.
To facilitate this, every reference to a terraform example should use test_structure.CopyTerraformFolderToTemp to create a copy of the example module in a temp directory. Then as the test runs, any stateful changes to the example module directory are isolated across tests, so that there’s no conflict on parallel runs.
Use TestStages for faster development
Use test stages
in the test code, unless you only have 1 or 2 steps in the test code (e.g. a plan
verification test).
It’s very tedious to build and deploy resources over and over when you only want to tweak a validation step. TestStages make it flexible and convenient to skip stages, making development much faster.
For each test stage you introduce, add a commented out series of os.Setenv
calls to make it convenient to skip stages
as you develop.
func TestJenkins(t *testing.T) {
t.Parallel()
// Uncomment the items below to skip certain parts of the test
//os.Setenv("SKIP_build_ami", "true")
//os.Setenv("SKIP_deploy_terraform", "true")
//os.Setenv("SKIP_validate", "true")
//os.Setenv("SKIP_cleanup", "true")
//os.Setenv("SKIP_cleanup_ami", "true")
defer test_structure.RunTestStage(t, "cleanup_ami", deleteAMI)
defer test_structure.RunTestStage(t, "cleanup", destroyInfra)
test_structure.RunTestStage(t, "build_ami", buildAMI)
test_structure.RunTestStage(t, "deploy_terraform", deployInfra)
test_structure.RunTestStage(t, "validate", validateInfra)
}
To use the stages, here’s an example workflow. The first time you run the test, you’ll want to skip only the cleanup
stages:
// Uncomment the items below to skip certain parts of the test
//os.Setenv("SKIP_build_ami", "true")
//os.Setenv("SKIP_deploy_terraform", "true")
//os.Setenv("SKIP_validate", "true")
os.Setenv("SKIP_cleanup", "true")
os.Setenv("SKIP_cleanup_ami", "true")
Let’s say building and deploying were successful, but validation failed. Since resources were not cleaned up, we can run
only the validate
stage. We skip the resource and time intensive build
and deploy
stages, and also continue to
skip the cleanup
stages.:
// Uncomment the items below to skip certain parts of the test
os.Setenv("SKIP_build_ami", "true")
os.Setenv("SKIP_deploy_terraform", "true")
//os.Setenv("SKIP_validate", "true")
os.Setenv("SKIP_cleanup", "true")
os.Setenv("SKIP_cleanup_ami", "true")
Once you’ve established that validation works, you can then run only the cleanup
stages as below. Your workflow may vary.
// Uncomment the items below to skip certain parts of the test
os.Setenv("SKIP_build_ami", "true")
os.Setenv("SKIP_deploy_terraform", "true")
os.Setenv("SKIP_validate", "true")
//os.Setenv("SKIP_cleanup", "true")
//os.Setenv("SKIP_cleanup_ami", "true")
When committing the final version of the test, all should be commented out so all stages run.
// Uncomment the items below to skip certain parts of the test
//os.Setenv("SKIP_build_ami", "true")
//os.Setenv("SKIP_deploy_terraform", "true")
//os.Setenv("SKIP_validate", "true")
//os.Setenv("SKIP_cleanup", "true")
//os.Setenv("SKIP_cleanup_ami", "true")
Setup and Teardown pattern
In some cases you will want to write a group of tests that use a common resource, such as a Docker image or VPC. In this case, you will want to setup the common resource once, run a bunch of tests, and then teardown the resource. To achieve this, you can follow the subtest pattern of Go.
Use table driven tests where possible to make the subtest routines maintainable. Briefly, this means that you group your test cases using a test struct that reflects the unique parameters of the test cases. Then you can conveniently loop over the test cases in parallel, taking advantage of uniformity and speed.
Note that the subtest pattern has gotchas when running tests in parallel:
-
The main test function will not wait for the subtest to run if it uses
t.Parallel
. To avoid this, you need to wrap the parallel subtests in a synchronous, blocking subtest. In the example below, thegroup
subtest is synchronous (no call tot.Parallel
) and thus the main function will wait until that test finishes. Thegroup
test does not finish until all the subtests it spawns are finished, even if they are non-blocking and parallel, and thus thetearDownVPC
call does not happen until all subtests are done. -
If you are using table driven tests, the range variable will be updated to the next iteration before it is used within the subtest. That is, in the example below, if we did not have the
testCase := testCase
line in the range block, thetestCase
reference used in the subtest after thet.Parallel
call will correspond to the lasttestCase
in thetestCases
list. To avoid this, we create a new variable in the scope of the range block so that it does not get updated during the loop.
Example:
func TestECS(t *testing.T) {
t.Parallel()
defer tearDownVPC()
deployVPC()
// Wrap the parallel tests in a synchronous test group to ensure that the main test function (the one calling
// `tearDownVPC` and `deployVPC`) waits until all the subtests are done before running the deferred function.
t.Run("group", func(t *testing.T) {
for _, testCase := range testCases {
// To avoid the range variable from getting updated in the parallel tests, we bind a new name that is within
// the scope of the for block.
testCase := testCase
t.Run(testCase.name, func(t *testing.T) {
t.Parallel()
testCase.testCode()
})
}
})
}