Pipelines integration

CI/CD is a crucial tool for ensuring the smooth iteration and consistent delivery of Infrastructure as Code (IaC) to production environments. By adopting CI/CD practices, teams can automate the process of integrating and testing changes made to IaC code, allowing for frequent and reliable updates. With CI/CD, each change to the IaC codebase triggers an automated build process, ensuring that any new additions or modifications are properly incorporated. This enables developers to catch errors and conflicts early, facilitating collaboration and reducing the likelihood of issues surfacing during deployment.

Gruntwork Pipelines is a framework that enables you to use your preferred CI tool to securely run an end-to-end pipeline for infrastructure code (Terraform) and app code (Docker or Packer). Rather than replace your existing CI/CD provider, Gruntwork Pipelines is designed to enhance the security of your existing tool. For more information please see the full pipelines documentation.

In the guide below, we walk through how to configure Gruntwork Pipelines in your CI/CD.

Set up machine user credentials

Get the machine user credentials from AWS

1. Log into the Security account in the AWS Console.

2. Go into IAM and find the ci-machine-user under Users.

3. Go to Security Credentials > Access Keys > Create Access Key.

4. Save these values as the AWS_ACCESS_KEY_ID and the AWS_SECRET_ACCESS_KEY Environment Variables in CircleCI.

   Env var name	Value
   AWS_ACCESS_KEY_ID	The Access Key generated for the machine user in the Security account.
   AWS_SECRET_ACCESS_KEY	The Secret Key generated for the machine user in the Security account.
   GITHUB_OAUTH_TOKEN	Enter the MachineUserGitHubPAT here. You can find this in reference-architecture-form.yml or in the shared account's Secrets Manager.

Verify: Testing an infrastructure change end to end

You can verify the pipeline by making a change to one of the modules. For example, follow the steps below to extend the number of replicas in the sample app:

1. Create a new branch in the infrastructure-live repo. git checkout -B add-replica-to-sample-app.
2. Open the file dev/us-west-2/dev/services/sample-app-frontend in your editor.
3. Change the input variable desired_number_of_tasks to 2.
4. Commit the change. git commit -a.
5. Push the branch to GitHub and open a PR. git push add-replica-to-sample-app
6. Login to CircleCI. Navigate to your infrastructure-live project.
7. Click on the new pipeline job for the branch add-replica-to-sample-app to see the build log.
8. Verify the plan. Make sure that the change corresponds to adding a new replica to the ECS service.
9. When satisfied with the plan, merge the PR into main.
10. Go back to the project and verify that a new build is started on the main branch.
11. Wait for the plan to finish. The build should hold for approval.
12. Approve the deployment by clicking Approve.
13. Wait for the apply to finish.
14. Login to the AWS console and verify the ECS service now has 2 replicas.

(Optional) Configure Slack notifications

Create a Slack App

1. Visit your apps on the Slack API website, and click Create New App.
2. Name your application (e.g., CircleCI or CircleCI-Pipeline).
3. Then select the Slack workspace in which to install this app.

Set Permissions

On the next page select the "Permissions" area, and add these 3 "scopes".

• chat:write
• chat:write.public
• files:write

Slack App Scopes

Install and Receive Token

Install the application into the Slack workspace and save your OAuth Access Token. This will be used in a CircleCI environment variable.

Slack OAuth Tokens

Slack OAuth Access Token

Choose a Slack channel to notify

1. Choose or create a Slack channel in your workspace to notify with pipeline updates.
2. Right-click the channel name. You'll see a context menu.
3. Select Copy link.
4. Extract the Channel ID from the URL copied. E.g., https://<org>.slack.com/archives/<CHANNEL_ID>

Create env vars on CircleCI

1. Login to CircleCI. Navigate to Project Settings -> Environment Variables.

2. Configure the following environment variables:

   Env var name	Value
   SLACK_ACCESS_TOKEN	The OAuth token acquired through the previous step.
   SLACK_DEFAULT_CHANNEL	If no channel ID is specified, the app will attempt to post here.