Reference Architecture - Application CI/CD with Kubernetes
On reference architecture, trying to build out a "CI/CD" based application flow using Git repo as source. We are using Kubernetes based installation, ECR repositories are set and deploying simple app or making application pointing out to docker image on ECR works fine. We have noticed that aws-sample-app is got pulled from gruntworks docker hub, so it is not actually pulled via CI/CD. **1-** What we are trying to do is deploy "multiple" K8S services using same application code that Environment variables can configure running service, yet how can Infrastruce code "tag" applications on the ECR docker repo on PR on infrastructure? For example instead of showing ECR containers can't we just show Git repository and build and push image on Infrastruce build time? **2-** I could not properly understand [ Infrastructure / CI / CD pipeline for app code instructions](https://github.com/gruntwork-io/terraform-aws-service-catalog/blob/5ec2974eed8883f0289a7c4b7a917fdc04c7a19a/examples/for-production/infrastructure-live/docs/04-configure-gw-pipelines.md?plain=1#L208), which seems unclear for me. It suggests copying _ci scripts to repository, yet Dockerfile context path is unclear and it is not "similar" to aws-sample-web-app, but I assume that there should be only 1 Dockerfile and path is actually if Dockerfile is in docker/Dockerfile Context_path is "docker". Can I deploy with support of docker-compose yml files? **3-** I am able to make a simple Github Actions pipeline and it is triggered as I copied _ci/app-templates/scripts to my app repo, still having trouble on GITHUB_OAUTH_TOKEN even I set GH_TOKEN on repository. I tried debugging with `gruntwork-install --binary-name "infrastructure-deployer" --repo "https:/URL_OF_MY_PRIVATE_GIT" --tag "GIT_RELEASE_TAG"` but gave Not Found even I have permission to repo and set` export GITHUB_OAUTH_TOKEN=XXX` --- <ins datetime="2022-04-25T20:27:02Z"> <p><a href="https://gruntwork.zendesk.com/agent/tickets/108497">Tracked in ticket #108497</a></p> </ins>
> 1- What we are trying to do is deploy "multiple" K8S services using same application code that Environment variables can configure running service, yet how can Infrastruce code "tag" applications on the ECR docker repo on PR on infrastructure? For example instead of showing ECR containers can't we just show Git repository and build and push image on Infrastruce build time? The core idea behind the pipeline in the Reference Architecture is as follows: - When a release tag is cut on the application repo, use the `infrastructure-deployer` CLI to build a new docker image in the ECS Deploy Runner and push to the corresponding ECR repo under the release tag for the application. This is what happens in the [build-docker-image.sh](https://github.com/gruntwork-io/terraform-aws-service-catalog/blob/master/examples/for-production/infrastructure-live/_ci/app-templates/scripts/build-docker-image.sh) CI script packaged with the Ref Arch. - Once a new image is built in ECR, you can then immediately deploy that to whatever environment you want it in by making an edit to the corresponding `terragrunt.hcl` file to use the new ECR tag. This step is handled by [the deploy-docker-image.sh](https://github.com/gruntwork-io/terraform-aws-service-catalog/blob/master/examples/for-production/infrastructure-live/_ci/app-templates/scripts/deploy-docker-image.sh) script of the Reference Architecture, which will make the edit and commit to `main` branch of `infrastructure-live`, triggering the deploy pipeline. > It suggests copying _ci scripts to repository, yet Dockerfile context path is unclear and it is not "similar" to aws-sample-web-app, but I assume that there should be only 1 Dockerfile and path is actually if Dockerfile is in docker/Dockerfile Context_path is "docker". I think what might be confusing you is that there are two `Dockerfile`s at play here: - One is for your application, which is presumably in the `docker` folder of your repo. - Another is for the CI server to get an image that has everything it needs setup. This is only needed if you are using Jenkins, and should be at the root of the repo if you want to use the `Jenkinsfile` directly. So the context path in the `constants.sh` script should point to the path to the application docker file. > Can I deploy with support of docker-compose yml files? We do not support `docker-compose.yml` files, nor does Kubernetes natively. At some point, you need to translate the `docker-compose.yml` file to Kubernetes manifests. That's what the `terragrunt.hcl` files in `infrastructure-live` correspond to, and the reason our pipeline in the Ref Arch work by making commits to `infrastructure-live`. > I am able to make a simple Github Actions pipeline and it is triggered as I copied _ci/app-templates/scripts to my app repo, still having trouble on GITHUB_OAUTH_TOKEN even I set GH_TOKEN on repository. I tried debugging with gruntwork-install --binary-name "infrastructure-deployer" --repo "https:/URL_OF_MY_PRIVATE_GIT" --tag "GIT_RELEASE_TAG" but gave Not Found even I have permission to repo and set export GITHUB_OAUTH_TOKEN=XXX The repo URL should be `https://github.com/gruntwork-io/terraform-aws-ci.git`. This code is downloading the `infrastructure-deployer` binary in our `terraform-aws-ci` repository. I'd need the full error logs to be sure, but on the surface based on the information you shared, I believe the `Not Found` is because the private git URL you are using doesn't have the `infrastructure-deployer` binary available on the release tag.