Upgrading from infrastructure-pipelines
This migration guide is for those using the earlier version of Gruntwork Pipelines, referred to as infrastructure-pipelines, and seeking to upgrade to the latest version of Gruntwork Pipelines. The latest version is referred to simply as "Pipelines" throughout this document.
What's new
For a detailed overview of changes and improvements in the latest version of Pipelines, refer to the deprecation notice.
Prerequisites
Before starting the migration process, ensure the following:
- mise is installed.
- AWS Administrator permissions (or equivalent) for all AWS accounts managed by your
infrastructure-liverepository. - A good understanding of Infrastructure as Code (IAC) or access to Gruntwork support.
Step 1: Ensure Machine Users are configured
Verify that appropriate tokens and secrets are configured to allow workflows to access the necessary infrastructure secrets. Detailed instructions can be found in the Machine Users guide.
Step 2: Update the bootstrap.yml workflow
Copy the bootstrap.yml file to the .github/workflows directory of your infrastructure-live repository. Overwrite any existing workflow file if necessary.
Review the README for an explanation of the workflow and required inputs. Ensure all <REPLACE> placeholders in the bootstrap.yml file are updated correctly.
Step 3: Run the bootstrap.yml workflow
Navigate to the Actions tab of your repository and locate the workflow labeled Infrastructure Live Root Bootstrap. Click the green Run workflow button, providing the necessary inputs as described in the Workflow Inputs section. Many inputs can be sourced from your accounts.yml file.
After execution, a new branch named bootstrap-repository and a corresponding pull request (PR) will be created in your repository.
Do not merge the PR immediately. Modify it as necessary to prevent conflicts with your existing codebase.
Modify the pull request
Access the bootstrap-repository branch, make the required changes to the PR, and ensure compatibility with your existing codebase.
git clone <your-repo-url>
cd <your-repo-name>
git checkout bootstrap-repository
The following sections outline the major changes introduced in this pull request and provide guidance on how to handle them appropriately for your repository.
If you encounter changes not covered here, or if any instructions are unclear, please contact Gruntwork support at support@gruntwork.io for assistance.
Reverting changes to your accounts.ymlfile
The bootstrap.yml workflow configures the accounts.yml file as if initializing a new repository. If you have previously customized this file, you will need to revert the changes made by the bootstrap.yml workflow to retain your existing configuration:
git checkout origin/main -- accounts.yml
Reverting the update of your multi_region_common.hcl file
The bootstrap.yml workflow sets your multi_region_common.hcl file as if you were setting up a new repository from scratch. You have likely already made changes to this file, and you will want to revert the changes that the bootstrap.yml workflow made to this file:
git checkout origin/main -- multi_region_common.hcl
Adding .mise.toml
To synchronize the versions of tools used locally with those used by Pipelines, Pipelines now integrates with the open-source tool mise. Mise enables you to define tool versions in a .mise.toml file, ensuring consistency between local environments and Pipelines.
The pull request includes a .mise.toml file that resembles the following:
[tools]
opentofu = "1.6.2"
terragrunt = "0.57.12"
awscli = "latest"
If the tools or versions specified in this file do not meet your requirements, you can modify them as needed. For instance, to pin the AWS CLI to a specific version, you can update the file as follows:
[tools]
opentofu = "1.6.2"
terragrunt = "0.57.12"
awscli = "2.15.44"
If you have not transitioned to OpenTofu yet, you can specify your current version of terraform in the .mise.toml file.
To install the dependencies locally, execute the following command:
mise install
You and your colleagues will now use this file to ensure synchronization between the versions of tools used locally and the versions used by Pipelines.
The .github/workflows/account-factory.yml workflow
If you are accustomed to an account-factory.yml workflow that uses a GitHub Actions form with multiple required inputs, note that this workflow now expects a JSON input string to configure the account request.
This adjustment enhances flexibility and repeatability when generating account requests.
A sample form is included in this initial pull request and can be found in the .github/workflows/account-factory-inputs.html file. You can modify the form as needed to align with your organization's requirements for account requests.
A common practice is to integrate an external ticketing system, such as JIRA, to collect the necessary information. This system can integrate with the GitHub API to automatically submit account request pull requests after review, or you can generate the JSON input string manually for use in the GitHub Actions workflow.
The state_bucket_pattern value in account.hcl
The account.hcl file previously included a state_bucket_name value to define the S3 bucket used for storing the state of resources in that account. This has been replaced by a state_bucket_pattern value, allowing roles interacting with S3 state to access a different state bucket per region. This change improves fault tolerance and facilitates data isolation by region.
Adding missing root-pipelines-plan and root-pipelines-apply roles
You may need to copy the github-actions-openid-connect-provider, root-pipelines-plan, and root-pipelines-apply folders from another account (such as the security account) to all accounts, depending on your current account structure.
These roles are required for Pipelines to plan and apply changes to your infrastructure. They must be present in every account that Pipelines interacts with via the infrastructure-live-root repository.
Due to the chicken-and-egg issue of needing roles present before they can be used, you must apply these changes manually:
Never modify Identity and Access Management (IAM) resources without thoroughly reviewing and understanding the changes being made.
Ensure that the permissions assigned to these roles are appropriate for your organization before applying them.
# # Ensure you have the necessary permissions for the account you are working in
cd <path-to-account>/_global/github-actions-openid-connect-provider
# If the OIDC provider already exists in your account but is not in the IaC state, you can import it
# Note: If `jq` is not installed, replace the code in `$()` below with the relevant account ID.
terragrunt import 'aws_iam_openid_connect_provider.github' "arn:aws:iam::$(aws sts get-caller-identity | jq -r '.Account'):oidc-provider/token.actions.githubusercontent.com"
terragrunt apply
# If you had to import this resource, you must remove it from the state file if it is defined elsewhere in your IaC setup to prevent potential errors later.
# For example, it may have previously been defined in the `github-oidc-role` folder:
cd ../github-oidc-role
terragrunt state rm 'aws_iam_openid_connect_provider.github[0]'
# Apply the changes to ensure that the `root-pipelines-plan` and `root-pipelines-apply` roles are present in the account
cd ../root-pipelines-plan
terragrunt apply
cd ../root-pipelines-apply
terragrunt apply
You must repeat this process for every existing account in your repository to ensure these roles are present. All accounts must have these roles for Pipelines to function properly.
Moving forward, the account baseline process will automatically ensure that these necessary roles are included in all accounts before Pipelines attempts to operate in them.
The pipelines.yml workflow
The pipelines.yml workflow serves as the primary workflow for planning and applying changes to your infrastructure. Previously, workflows were often more complex and relied on interactions with a secondary infrastructure-pipelines repository. This migration guide addresses these changes and introduces a simplified approach.
The logic previously managed by dispatching workflows to a secondary infrastructure-pipelines repository has been replaced with a shared workflow located in the Gruntwork-managed pipelines-workflows repository. By default, any repository using Pipelines will reference this shared workflow.
Repositories using infrastructure-live now operate within their own context. Secrets and role assumptions are specific to the repository executing the workflow, eliminating the need for a secondary repository. This adjustment simplifies infrastructure management at scale and offers numerous benefits over the previous method. To learn more, see the deprecation notice here.
Please ensure you understand these changes. For questions, contact Gruntwork support.
The .gitignore file
Previously, .terraform.lock.hcl was included in the .gitignore file to reduce confusion caused by platform-specific errors (e.g., between MacOS and Linux). However, this entry has been removed to align with best practices. Documentation has been added to the .gitignore file to explain the reasoning and how to prevent issues in your workflows.
Ensure that no other unintended changes have occurred in your .gitignore file. Reproduce custom adjustments if necessary.
The .gruntwork/config.yml file
The .gruntwork/config.yml file has expanded responsibilities. It now includes multiple configurations dynamically altering Pipelines workflows.
Notably, Pipelines CLI, Terragrunt, and OpenTofu/Terraform versions are no longer specified here but instead in the .mise.toml file and the pipelines-workflows repository. This approach ensures consistent tool versions across local environments, workstations, and Pipelines.
OpenTofu by default
Gruntwork Pipelines now defaults to OpenTofu. If you wish to continue using the Terraform binary, you can configure the tf-binary value in .gruntwork/config.yml to specify your preference.
The tags.yml file
Depending on when your infrastructure-live repository was configured, tags.yml files may not exist at the root of your repository or account-specific folders. These files are essential for ensuring all resources provisioned via IaC are tagged appropriately. Proper tagging is a critical best practice for cloud infrastructure, aiding cost optimization and security (e.g., when using Attribute-Based Access Control).
Refer to the Tagging section in the updated README.md included in the pull request generated by the bootstrap.yml workflow for more information.
Removal of source_base_url from terragrunt.hcl files
Earlier versions of infrastructure-live repositories often used a source_base_url local variable in terragrunt.hcl files to centralize module sourcing. This pattern has been deprecated due to its complexity and the difficulty of managing configuration isolation.
If you have a source_base_url local variable in your terragrunt.hcl files, you will need to remove it and update the source values in your terraform configuration blocks to reflect the underlying module source.
You can do this by searching for source_base_url in your editor, taking note of every location where this pattern is used, then finding the referenced include (usually the equivalent _envcommon folder for the folder you are looking at), and updating the source value to reflect the new location.
The README.md file
The README.md file has been updated to provide additional context about the repository structure and usage. Review the changes to ensure it adequately explains how the repository is organized and its intended use.
Step 4: Merge the pull request
Once you've made the required changes to the pull request and verified compatibility with your existing codebase, merge the pull request.
Include [skip ci] in the commit message to prevent Pipelines from running unnecessary workflows.
If you need to revert these changes later, open the merged pull request in your browser, click the Revert button, and create a revert pull request. Be sure to include [skip ci] in the commit message for the revert as well.
Step 5: Cleanup
With the migration complete, you should clean up unnecessary resources to streamline your repository and reduce clutter.
If you're uncertain about the new setup and want a fallback option, you can defer cleanup until you're confident everything is functioning as expected.
Remove unnecessary files
Search for and remove outdated files or folders that are no longer needed, such as:
_envcommon/landingzone/pipelines-pre-auth-role.hcl_envcommon/landingzone/central-pipelines-plan-role.hcl_envcommon/landingzone/central-pipelines-apply-role.hcl_envcommon/landingzone/team-pipelines-plan-role.hcl_envcommon/landingzone/team-pipelines-apply-role.hcl_envcommon/landingzone/pipelines-policy-plan-update-role.hcl_envcommon/landingzone/pipelines-policy-apply-update-role.hcl_envcommon/landingzone/github-oidc-role.hcl
Some folders that you should search across your repository for removal include:
pipelines-pre-auth-roleteam-pipelines-plan-roleteam-pipelines-apply-rolepipelines-pre-auth-rolecentral-pipelines-plan-rolecentral-pipelines-apply-rolepipelines-policy-plan-update-rolepipelines-policy-apply-update-rolegithub-oidc-role
It is very likely that you will not have all of these files in your infrastructure-live repository, but you should search for them to ensure that you are not leaving any unnecessary files in your repository.
Consult Gruntwork support if you're unsure about residual files.
Delete old tokens and secrets
Remove any tokens or secrets used with the previous infrastructure-pipelines setup. For a list of old tokens, refer to Machine Users Documentation.
Archive the infrastructure-pipelines repository
The infrastructure-pipelines repository is no longer needed, and should be archived to prevent accidental use. Refer to GitHub's archiving documentation.
Note that this is a reversible process, which is advisable in case you need to revert back to the old setup for any reason. If you would prefer to permanently delete the repository instead, you can refer to the GitHub documentation on deleting the repository.
Conclusion 🎉
You have successfully migrated your repository from infrastructure-pipelines! We recommend performing small tests to ensure stability and familiarize yourself with the new setup.
Create a new pull request that makes a small change to your infrastructure code (adding a comment to a terragrunt.hcl file will trigger Pipelines), and ensure that the plan and apply workflows run as expected, that you know where the logs will appear, and that you have a good understanding of how to troubleshoot any issues that may arise.
If you encounter any issues or have suggestions for improving this guide, please contact Gruntwork support or contribute to the community guide.