In the Reference Architecture, why is there an ECS Deploy Runner in each account? Is this DRY?
A customer asked if having an `ecs-deploy-runner` in each account is a violation of the DRY (don't repeat yourself) principle: > This is kind of an abstract, strategy question but here goes. With our refarch, we have a number of DRY violations (in particular, ecs-deploy-runner s in each account). Is it generally advisable to identify these things, then introduce a higher-level module (i.e. by wrapping the Gruntwork one) with all the commonality to reduce the size of the leaf terragrunt.hcl files?
## Having one ECS Deploy Runner in each account is intended The ECS Deploy Runner (EDR or `ecs-deploy-runner`) actually has two purposes: 1. During the Ref Arch deployment, performed by a Gruntwork engineer, the EDR is intentionally spun up in each account. Once up, it is the responsibility of the EDR to run `terragrunt run-all apply` on that particular account's infrastructure-live folder (e.g., `dev`) in order to provision the resources defined in the scaffolded code. So, it's the EDR that is actually responsible for deploying the generated code that defined a Ref Arch 2. Once the Ref Arch is handed off, the EDR intentionally remains in each account because it is what runs the CI/CD operations (running `terragrunt plan` when you push a feature branch, or `terragrunt apply` when you merge to main) and it is a core component of what makes our product unique. ## EDR is concerned with locking down CI/CD operations for enhanced security In a traditional infrastructure CI/CD setup, you are bound to give unlimited admin privileges to the CI server so that it can deploy arbitrary infrastructure. However, this has many risks, the primary one being a user with write access can open a PR that does malicious things in the CI pipeline by editing the CI config. For example, a user could modify one of the steps in `.circleci/config.yml` to run the command `env`, which would dump all environment variables to `stdout`. (However it appears that CircleCI is smart enough to obfuscate credentials which are defined as Environment Variables in the Project Settings.) To protect against this, the `ecs-deploy-runner` employs a limited action model, where the AWS credentials provided to the CI server is only allowed to trigger specific actions, one of them being running `terraform apply` on the latest infra code. ## DRYing up EDR configurations By nature, the `ecs-deploy-runner` is a commodity in that there isn’t much difference between all the accounts. The biggest difference is actually in the `shared` account, where the `ecs-deploy-runner` is configured to also build docker images and AMIs. However, outside of that all the other app accounts have the exact same `ecs-deploy-runner` configuration. In that regard, DRY-ing up the configurations that are currently nearly identical across accounts would reduce complexity by making it less likely to forget to update all the `ecs-deploy-runner`s. That said, the challenge is DRY-ing up the configuration in a satisfactory way that doesn’t inadvertently introduce some other bug. For what it's worth, we are really close to offering something better in the `imports` feature: https://github.com/gruntwork-io/terragrunt/issues/1566