Skip to main content

Update Grouping & Pull Request Strategy

Overview​

Enterprise IaC repositories often reuse modules extensively across a single repository. As a result, updating a dependent module version can require changes in multiple references across the codebase. Unlike application codebases in environments like Java or Node.js—where a single version bump typically involves updating centralized files such as gradle.build or package.json—IaC repositories frequently repeat environments and dependencies across folders. This repetition creates a challenge: how to group dependency updates into pull requests for optimal efficiency and maintainability?

The simplest approach is to create one pull request for each dependency update. While simple, this method can quickly become overwhelming at scale. Patcher supports this approach but also offers more flexible options for grouping updates. Pull requests can be consolidated based on the following strategies:

  • Full-consolidation: A single pull request updates all dependencies across the repository.
  • No-consolidation: A separate pull request is created for each individual update.
  • Dependency-only consolidation: A single pull request is created for each dependency, updating it across all environments.
  • Environment-only consolidation: A single pull request is created for each environment, updating all dependencies within that environment.
  • (Environment x Dependency) consolidation: A single pull request is created for each dependency within each environment.

Grouping examples​

To demonstrate these strategies, consider the following example repository:

/dev/unit1/terragrunt.hcl -> dependency1@1.0.0
/dev/unit2/terragrunt.hcl -> dependency1@1.0.0
/dev/unit3/terragrunt.hcl -> dependency2@1.0.0
/prod/unit1/terragrunt.hcl -> dependency1@1.0.0
/prod/unit2/terragrunt.hcl -> dependency1@1.0.0
/prod/unit3/terragrunt.hcl -> dependency2@1.0.0
/prod/unit4/terragrunt.hcl -> dependency3@1.0.0

Assuming newer versions are available for all three dependencies, the strategies would result in:

  • Full-consolidation: One pull request updating all seven units.
  • No-consolidation: Seven separate pull requests, one per unit.
  • Dependency-only consolidation: Three pull requests—one for each dependency. For example, dependency1 would be updated across both dev and prod.
  • Environment-only consolidation: Two pull requests—one for all updates in dev and one for all updates in prod.
  • (Environment x Dependency) consolidation: Five pull requests—two for updates in dev and three for updates in prod.

Terminology​

  • unit: A folder containing a terragrunt.hcl file, representing a single OpenTofu state file. A unit may reference one or more units as dependencies.

  • dependency (or target): An OpenTofu module referenced using a ref (typically a full source path and version) in a unit. Patcher interprets semantic versioning for dependency updates.

  • environment: A logical grouping of infrastructure representing application stages, such as dev or prod. Environments are generally organized as folders in the repository and include multiple units and dependencies.

  • update: The action of modifying a dependency reference to use a newer version and accommodating any associated breaking changes.

    info

    As of November 2024, Patcher recognizes environments using folder groupings matched with glob patterns. For example, dev may correspond to dev-* folders and 'prod' to prod-* folders. A more sophisticated environment definition using HCL syntax (similar to Pipelines) is planned for future releases. Let us know if this capability is important for your use case.

Implementation discussion​

In CI workflows, pull requests are typically generated by first running a patcher report to identify updates, followed by patcher update to apply those updates. Patcher does not include a single option to specify grouping strategies by name. Instead, grouping is implemented through combinations of report and update command flags.

Patcher report​

The patcher report command accepts the --include-dirs flag, which filters updates based on matching glob patterns. This allows developers to limit updates to specific environments. By running patcher report multiple times with different --include-dirs values, you can create distinct workflows for each environment. Patcher report outputs in JSON which can be inspected or iterated over to achieve desired behaviors.

The patcher report output is JSON-formatted and can be inspected or iterated for further customization.

Patcher update​

The patcher update command accepts a --target flag, which specifies one or more dependencies to update. By running patcher update with different --target values, you can control which dependencies are included in each pull request.

Implementation walkthrough​

Full-consolidation​

For full consolidation, omit the --include-dirs and --target arguments. This approach generates a single pull request containing all updates.

Pseudocode:

run patcher report
run patcher update

No-consolidation​

To create one pull request per update, use the plan output of patcher report and iterate over each dependency and its instances.

Pseudocode:

run patcher report --output-plan plan.json
for each dependency in plan.json
    for each usage of the dependency:
        cd to the directory of the usage, e.g. (dirname usage.source.file)
        run patcher update --target $dependency.org/$dependency.repo/$dependency.module

Dependency-only consolidation​

To group by dependency, run patcher update for each target without filtering environments.

Pseudocode:

run patcher report without an include-dirs argument
for each $target in output 
    run patcher update --target=$target

Environment-only consolidation​

To group updates by environment, use --include-dirs to filter by environment and run patcher update without specifying targets.

Pseudocode:

for each $environment
    run patcher report -include-dirs=$environment
    run patcher update without any target argument

(Environment x Dependency) consolidation​

To create pull requests for each dependency within specific environments, combine --include-dirs with --target.

Pseudocode:

for each $environment e.g., glob patterns like dev-* or prod-*)
  run patcher report --include-dirs=$environment
  for each $target output 
    run patcher update --target=$target