Terragrunt Github Pipeline with multiple environments and dependencies
Hi all :wave: I'm setting up a new Terragrunt repository and have few questions specially regarding the best practices around the pipeline. I'll be using Github Actions. My current Terragrunt structure looks like this: ``` . ├── config │ ├── _envcommon │ │ ├── README.md │ │ ├── s3.hcl │ │ └── s3-logging-bucket.hcl │ ├── Makefile ├── environments │ ├── dev │ │ ├── ap-southeast-2 │ │ │ ├── region.hcl │ │ │ ├── s3 │ │ │ │ ├── Makefile │ │ │ │ └── terragrunt.hcl │ │ │ └── s3-logging-bucket │ │ │ ├── Makefile │ │ │ └── terragrunt.hcl │ │ ├── env.hcl │ │ └── Makefile │ ├── prod │ │ └── env.hcl │ └── staging │ └── env.hcl ├── modules ├── README.md └── terragrunt.hcl ``` The **main** reason I'm using Terragrunt, is because after few years mainly using Terraform I ended up learning the hard way that Terraform sucks when there are multiple modules, state files and dependencies between them. Looks like [I cannot use `run-all` with Terragrunt](https://github.com/gruntwork-io/terragrunt/issues/720#issuecomment-497888756) when there are dependencies between modules. In that case, I'll have to make my CI to enter in each `environment/**/ap-southeast-2/**/` folder and run the commands there. However by doing that, I'll be creating lots and lots of duplicated files in the `.github/workflow` directory for each of the resources. Github Actions will need to perform the following: ``` [...] cd environment/dev/ap-southeast-2/s3 terragrunt plan [...] [...] cd environment/dev/ap-southeast-2/s3-logging-bucket terragrunt plan [...] [...] cd environment/dev/ap-southeast-2/vpc terragrunt plan [...] etc ``` Wondering if there are better ways to achieve the same result? Any suggestions are more than welcome. Cheers! :smile: --- <ins datetime="2023-05-10T02:44:54Z"> <p><a href="https://support.gruntwork.io/hc/requests/110155">Tracked in ticket #110155</a></p> </ins>
> @lpossamai we have a similar issue when trying to run `terragrunt run-all init` and `terragrunt run-all plan` against all of our terragrunt directories. We're using terragrunt in a CICD pipeline via GitHub Actions and after a few iterations, we got around the output dependency issue by introducing the mock outputs, but now we get something like `terragrunt Underlying error: invalid character ':' after top-level value` when trying to do a `run-all plan`. Have you managed to find a workaround? In the interim, we've created a Makefile and are now running the int, plan and apply against the first directory then moving onto the second directory. But this is not ideal. Ideally, we want a separate task in GitHub Actions to run a `terragrunt run-all init` then a separate task to run a `terragrunt run-all plan` and then finally a separate task to run a `terragrunt run-all apply`. Any advice? Hey! The only solution I found was to use [Atlantis](https://www.runatlantis.io/) or [Terrateam](https://terrateam.io/). I've been using Terrateam and the community there is great and really help you out! I strongly suggest you try them :) It's been working great for me! I even got in touch with [antonbabenko](https://github.com/antonbabenko) to ask for some advise but he also mentioned the two solutions above. Good luck!