How Gruntwork Pipelines works in a multi-account, multi-region model with terragrunt?
Hello, I am trying to understand how the gruntwork pipelines framework will interact with an AWS multi-account, multi-region topology. We are using `terragrunt` to manage our infrastructure similar to the one you provide in the [infrastructure-live folder](https://github.com/gruntwork-io/terraform-aws-service-catalog/tree/master/examples/for-production/infrastructure-live) of your terraform-aws-service-catalog repository. We created a new Shared AWS Account to deploy the secrets required to communicate with our VCS. Based on the diagram [here](https://docs.gruntwork.io/assets/images/landing-zone-ref-arch-08ff1641db41e41a0ad9d50c34a4a61b.png), I concluded that the "terraform/terragrunt executors" (i.e., `ecs-deploy-runner` tasks in `ECS`) should be deployed in all the AWS Accounts (dev, prod, security, etc...) for all regions (using multi-region modules as defined [here](https://github.com/gruntwork-io/terraform-aws-service-catalog/blob/master/examples/for-production/infrastructure-live/_envcommon/mgmt/ecs-deploy-runner.hcl#L38-L60)). > Note: The VPC Management part, though, as a [dependency](https://github.com/gruntwork-io/terraform-aws-service-catalog/blob/master/examples/for-production/infrastructure-live/_envcommon/mgmt/ecs-deploy-runner.hcl#L24-L32) to this multi-region module and therefore still need to be deployed in all the regions. This decentralized model sounds great but leads me to some questions. Considering that we have the live repository with `terragrunt` to represent our entire infrastructure, I wondered how the CI server would identify which `lambda invoker` function / `ecs-deploy-runner` (in which region, which account)? In our case, we plan to use GitLab and the OIDC integration to avoid storing secrets. How can I know in advance which IAM role (allowing me to invoke the lambda) I need to assume to invoke the lambda invoker? Let's say we have the following structure: ```bash . └── live ├── sandbox-account-backend # <---- Where my microservices are *deployed* │ ├── _global │ │ └── iam │ ├── eu-west-1 │ │ ├── eks │ │ └── mgmt │ │ ├── ecs-deploy-runner # <---- My multi-region module for this account │ │ └── networking │ │ └── vpc │ └── eu-west-3 │ ├── eks │ └── mgmt │ └── networking # <---- Still needed as we don't have a "multi-region" module for VPC Mgmt │ └── vpc ├── sandbox-account-networking │ ├── _global │ │ ├── dns │ │ └── iam │ ├── eu-west-1 │ │ ├── apigateway # <---- Where my microservices are *exposed* │ │ └── mgmt │ │ ├── ecs-deploy-runner │ │ └── networking │ │ └── vpc │ └── eu-west-3 │ ├── apigateway │ └── mgmt │ └── networking │ └── vpc └── shared └── eu-west-1 ├── _regional │ └── ecr-repos └── mgmt ├── ecs-deploy-runner └── networking ``` Let's consider I create a new PR modifying the EKS cluster in `live/sandbox-account-backend/eu-west-1/eks` and the associated apigateway in the networking account in `live/sandbox-account-networking/eu-west-1/apigateway`. Which `ecs-deploy-runner` task will be called by GitLab CI? And if we have a cross-account dependency (e.g., apigateway depends on eks), how can we ensure the (cross-account) order will be insured? --- <ins datetime="2022-05-12T11:51:35Z"> <p><a href="https://support.gruntwork.io/hc/requests/108582">Tracked in ticket #108582</a></p> </ins>
The primary way to handle this is by routing with a lookup table and using heuristics with the folder structure to guide you on which entry in the table to get. For example, in the Reference Architecture, we handle this by checking for all the terragrunt folders that changed (in [this line](https://github.com/gruntwork-io/terraform-aws-service-catalog/blob/master/examples/for-production/infrastructure-live/_ci/scripts/deploy-infra.sh#L105-L108)), and then for each one, extracting out [the first folder in the path](https://github.com/gruntwork-io/terraform-aws-service-catalog/blob/master/examples/for-production/infrastructure-live/_ci/scripts/deploy-infra.sh#L66-L69) and using that to lookup the AWS Account ID that corresponds to the folder by matching the folder name to the entry in the accounts.json file, and then using the ID to construct the IAM Role ARN to assume and assume that (see [this function](https://github.com/gruntwork-io/terraform-aws-service-catalog/blob/master/examples/for-production/infrastructure-live/_ci/scripts/helpers.sh#L10-L22)). We are not very familiar with the GitLab OIDC integration for assuming a role, but presumably you can use similar logic to identify the IAM Role you need to assume in the workflow, and then extract that out as a step output to feed into the OIDC mechanism. Hope this helps!