Kubernetes Service
Overview
This service contains Terraform code to deploy your web application containers using the k8-service Gruntwork Helm Chart on to Kubernetes following best practices.
If you want to deploy third-party applications already packaged as Helm Charts, such as those available in bitnami, see the helm-service
module.
Kubernetes Service architecture
Features
- Deploy your application containers on to Kubernetes
- Zero-downtime rolling deployments
- Auto scaling and auto healing
- Configuration management and Secrets management
- Ingress and Service endpoints
- Service discovery with Kubernetes DNS
- Managed with Helm
Learn
This repo is a part of the Gruntwork Service Catalog, a collection of reusable, battle-tested, production ready infrastructure code. If you’ve never used the Service Catalog before, make sure to read How to use the Gruntwork Service Catalog!
Under the hood, this is all implemented using Terraform modules from the Gruntwork helm-kubernetes-services repo. If you are a subscriber and don’t have access to this repo, email support@gruntwork.io.
Core concepts
-
Kubernetes core concepts: learn about Kubernetes architecture (control plane, worker nodes), access control (authentication, authorization, resources (pods, controllers, services, config, secrets), and more.
-
Kubernetes in Action: the best book we’ve found for getting up and running with Kubernetes.
Repo organization
- modules: the main implementation code for this repo, broken down into multiple standalone, orthogonal submodules.
- examples: This folder contains working examples of how to use the submodules.
- test: Automated tests for the modules and examples.
Deploy
Non-production deployment (quick start for learning)
If you just want to try this repo out for experimenting and learning, check out the following resources:
- examples/for-learning-and-testing folder: The
examples/for-learning-and-testing
folder contains standalone sample code optimized for learning, experimenting, and testing (but not direct production usage).
Production deployment
If you want to deploy this repo in production, check out the following resources:
- examples/for-production folder: The
examples/for-production
folder contains sample code optimized for direct usage in production. This is code from the Gruntwork Reference Architecture, and it shows you how we build an end-to-end, integrated tech stack on top of the Gruntwork Service Catalog.
Sample Usage
- Terraform
- Terragrunt
# ------------------------------------------------------------------------------------------------------
# DEPLOY GRUNTWORK'S K8S-SERVICE MODULE
# ------------------------------------------------------------------------------------------------------
module "k_8_s_service" {
source = "git::git@github.com:gruntwork-io/terraform-aws-service-catalog.git//modules/services/k8s-service?ref=v0.116.1"
# ----------------------------------------------------------------------------------------------------
# REQUIRED VARIABLES
# ----------------------------------------------------------------------------------------------------
# The name of the application (e.g. my-service-stage). Used for labeling
# Kubernetes resources.
application_name = <string>
# The Docker image to run.
container_image = <object(
repository = string
tag = string
pull_policy = string
)>
# The port number on which this service's Docker container accepts incoming
# traffic.
container_port = <number>
# The number of Pods to run on the Kubernetes cluster for this service.
desired_number_of_pods = <number>
# The Kubernetes Namespace to deploy the application into.
namespace = <string>
# ----------------------------------------------------------------------------------------------------
# OPTIONAL VARIABLES
# ----------------------------------------------------------------------------------------------------
# Map of additional ports to expose for the container. The key is the name of
# the port and value contains port number and protocol.
additional_ports = null
# A list of ACM certificate ARNs to attach to the ALB. The first certificate
# in the list will be added as default certificate.
alb_acm_certificate_arns = []
# The number of consecutive health check successes required before considering
# an unhealthy target healthy.
alb_health_check_healthy_threshold = 2
# Interval between ALB health checks in seconds.
alb_health_check_interval = 30
# URL path for the endpoint that the ALB health check should ping. Defaults to
# /.
alb_health_check_path = "/"
# String value specifying the port that the ALB health check should probe. By
# default, this will be set to the traffic port (the NodePort or port where
# the service receives traffic). This can also be set to a Kubernetes named
# port, or direct integer value. See
# https://kubernetes-sigs.github.io/aws-load-balancer-controller/v2.3/guide/ingress/annotations/#healthcheck-port
# for more information.
alb_health_check_port = "traffic-port"
# Protocol (HTTP or HTTPS) that the ALB health check should use to connect to
# the application container.
alb_health_check_protocol = "HTTP"
# The HTTP status code that should be expected when doing health checks
# against the specified health check path. Accepts a single value (200),
# multiple values (200,201), or a range of values (200-300).
alb_health_check_success_codes = "200"
# The timeout, in seconds, during which no response from a target means a
# failed health check.
alb_health_check_timeout = 10
# The Docker image to use for the canary. Required if
# desired_number_of_canary_pods is greater than 0.
canary_image = null
# Allow deletion of new resources created in this upgrade when upgrade fails.
cleanup_on_fail = null
# Kubernetes ConfigMaps to be injected into the container. Each entry in the
# map represents a ConfigMap to be injected, with the key representing the
# name of the ConfigMap. The value is also a map, with each entry
# corresponding to an entry in the ConfigMap, with the key corresponding to
# the ConfigMap entry key and the value corresponding to the environment
# variable name.
configmaps_as_env_vars = {}
# Kubernetes ConfigMaps to be injected into the container as volume mounts.
# Each entry in the map represents a ConfigMap to be mounted, with the key
# representing the name of the ConfigMap and the value as a map containing
# required mountPath (file path on the container to mount the ConfigMap to)
# and optional subPath (sub-path inside the referenced volume).
configmaps_as_volumes = {}
# The protocol on which this service's Docker container accepts traffic. Must
# be one of the supported protocols:
# https://kubernetes.io/docs/concepts/services-networking/service/#protocol-support.
container_protocol = "TCP"
# The map that lets you define Kubernetes resources you want installed and
# configured as part of the chart.
custom_resources = {}
# A list of custom Deployment annotations, to add to the Helm chart. See:
# https://kubernetes.io/docs/concepts/overview/working-with-objects/annotations/
deployment_annotations = {}
# The number of canary Pods to run on the Kubernetes cluster for this service.
# If greater than 0, you must provide var.canary_image.
desired_number_of_canary_pods = 0
# The domain name for the DNS A record to bind to the Ingress resource for
# this service (e.g. service.foo.com). Depending on your external-dns
# configuration, this will also create the DNS record in the configured DNS
# service (e.g., Route53).
domain_name = null
# The TTL value of the DNS A record that is bound to the Ingress resource.
# Only used if var.domain_name is set and external-dns is deployed.
domain_propagation_ttl = null
# Configuration for using the IAM role with Service Accounts feature to
# provide permissions to the applications. This expects a map with two
# properties: `openid_connect_provider_arn` and `openid_connect_provider_url`.
# The `openid_connect_provider_arn` is the ARN of the OpenID Connect Provider
# for EKS to retrieve IAM credentials, while `openid_connect_provider_url` is
# the URL. Leave as an empty string if you do not wish to use IAM role with
# Service Accounts.
eks_iam_role_for_service_accounts_config = {"openid_connect_provider_arn":"","openid_connect_provider_url":""}
# Whether or not to enable liveness probe. Liveness checks indicate whether or
# not the container is alive. When these checks fail, the cluster will
# automatically rotate the Pod.
enable_liveness_probe = false
# Whether or not to enable readiness probe. Readiness checks indicate whether
# or not the container can accept traffic. When these checks fail, the Pods
# are automatically removed from the Service (and added back in when they
# pass).
enable_readiness_probe = false
# A map of environment variable name to environment variable value that should
# be made available to the Docker container.
env_vars = {}
# How the service will be exposed in the cluster. Must be one of `external`
# (accessible over the public Internet), `internal` (only accessible from
# within the same VPC as the cluster), `cluster-internal` (only accessible
# within the Kubernetes network), `none` (deploys as a headless service with
# no service IP).
expose_type = "cluster-internal"
# A boolean that indicates whether the access logs bucket should be destroyed,
# even if there are files in it, when you run Terraform destroy. Unless you
# are using this bucket only for test purposes, you'll want to leave this
# variable set to false.
force_destroy_ingress_access_logs = false
# The version of the k8s-service helm chart to deploy.
helm_chart_version = "v0.2.18"
# Configure the Horizontal Pod Autoscaler (HPA) information for the associated
# Deployment. HPA is disabled when this variable is set to null. Note that to
# use an HPA, you must have a corresponding service deployed to your cluster
# that exports the metrics (e.g., metrics-server
# https://github.com/kubernetes-sigs/metrics-server).
horizontal_pod_autoscaler = null
# An object defining the policy to attach to `iam_role_name` if the IAM role
# is going to be created. Accepts a map of objects, where the map keys are
# sids for IAM policy statements, and the object fields are the resources,
# actions, and the effect ("Allow" or "Deny") of the statement. Ignored if
# `iam_role_arn` is provided. Leave as null if you do not wish to use IAM role
# with Service Accounts.
iam_policy = null
# Whether or not the IAM role passed in `iam_role_name` already exists. Set to
# true if it exists, or false if it needs to be created. Defaults to false.
iam_role_exists = false
# The name of an IAM role that will be used by the pod to access the AWS API.
# If `iam_role_exists` is set to false, this role will be created. Leave as an
# empty string if you do not wish to use IAM role with Service Accounts.
iam_role_name = ""
# Set to true if the S3 bucket to store the Ingress access logs is managed
# external to this module.
ingress_access_logs_s3_bucket_already_exists = false
# The name to use for the S3 bucket where the Ingress access logs will be
# stored. If you leave this blank, a name will be generated automatically
# based on var.application_name.
ingress_access_logs_s3_bucket_name = ""
# The prefix to use for ingress access logs associated with the ALB. All logs
# will be stored in a key with this prefix. If null, the application name will
# be used.
ingress_access_logs_s3_prefix = null
# A list of custom ingress annotations, such as health checks and TLS
# certificates, to add to the Helm chart. See:
# https://kubernetes-sigs.github.io/aws-load-balancer-controller/v2.4/guide/ingress/annotations/
ingress_annotations = {}
# The protocol used by the Ingress ALB resource to communicate with the
# Service. Must be one of HTTP or HTTPS.
ingress_backend_protocol = "HTTP"
# When true, HTTP requests will automatically be redirected to use SSL
# (HTTPS). Used only when expose_type is either external or internal.
ingress_configure_ssl_redirect = true
# Assign the ingress resource to an IngressGroup. All Ingress rules of the
# group will be collapsed to a single ALB. The rules will be collapsed in
# priority order, with lower numbers being evaluated first.
ingress_group = null
# A list of maps of protocols and ports that the ALB should listen on.
ingress_listener_protocol_ports = [{"port":80,"protocol":"HTTP"},{"port":443,"protocol":"HTTPS"}]
# Path prefix that should be matched to route to the service. For Kubernetes
# Versions <1.19, Use /* to match all paths. For Kubernetes Versions >=1.19,
# use / with ingress_path_type set to Prefix to match all paths.
ingress_path = "/"
# The path type to use for the ingress rule. Refer to
# https://kubernetes.io/docs/concepts/services-networking/ingress/#path-types
# for more information.
ingress_path_type = "Prefix"
# Set to true if the Ingress SSL redirect rule is managed externally. This is
# useful when configuring Ingress grouping and you only want one service to be
# managing the SSL redirect rules. Only used if ingress_configure_ssl_redirect
# is true.
ingress_ssl_redirect_rule_already_exists = false
# Whether or not the redirect rule requires setting path type. Set to true
# when deploying to Kubernetes clusters with version >=1.19. Only used if
# ingress_configure_ssl_redirect is true.
ingress_ssl_redirect_rule_requires_path_type = true
# Controls how the ALB routes traffic to the Pods. Supports 'instance' mode
# (route traffic to NodePort and load balance across all worker nodes, relying
# on Kubernetes Service networking to route to the pods), or 'ip' mode (route
# traffic directly to the pod IP - only works with AWS VPC CNI). Must be set
# to 'ip' if using Fargate. Only used if expose_type is not cluster-internal.
ingress_target_type = "instance"
# Seconds to wait after Pod creation before liveness probe has any effect. Any
# failures during this period are ignored.
liveness_probe_grace_period_seconds = 15
# The approximate amount of time, in seconds, between liveness checks of an
# individual Target.
liveness_probe_interval_seconds = 30
# URL path for the endpoint that the liveness probe should ping.
liveness_probe_path = "/"
# Port that the liveness probe should use to connect to the application
# container.
liveness_probe_port = 80
# Protocol (HTTP or HTTPS) that the liveness probe should use to connect to
# the application container.
liveness_probe_protocol = "HTTP"
# The minimum number of pods that should be available at any given point in
# time. This is used to configure a PodDisruptionBudget for the service,
# allowing you to achieve a graceful rollout. See
# https://blog.gruntwork.io/avoiding-outages-in-your-kubernetes-cluster-using-poddisruptionbudgets-ef6a4baa5085
# for an introduction to PodDisruptionBudgets.
min_number_of_pods_available = 0
# After this number of days, Ingress log files should be transitioned from S3
# to Glacier. Set to 0 to never archive logs.
num_days_after_which_archive_ingress_log_data = 0
# After this number of days, Ingress log files should be deleted from S3. Set
# to 0 to never delete logs.
num_days_after_which_delete_ingress_log_data = 0
# Override any computed chart inputs with this map. This map is shallow merged
# to the computed chart inputs prior to passing on to the Helm Release. This
# is provided as a workaround while the terraform module does not support a
# particular input value that is exposed in the underlying chart. Please
# always file a GitHub issue to request exposing additional underlying input
# values prior to using this variable.
override_chart_inputs = {}
# A list of custom Pod annotations, to add to the Helm chart. See:
# https://kubernetes.io/docs/concepts/overview/working-with-objects/annotations/
pod_annotations = {}
# Seconds to wait after Pod creation before liveness probe has any effect. Any
# failures during this period are ignored.
readiness_probe_grace_period_seconds = 15
# The approximate amount of time, in seconds, between liveness checks of an
# individual Target.
readiness_probe_interval_seconds = 30
# URL path for the endpoint that the readiness probe should ping.
readiness_probe_path = "/"
# Port that the readiness probe should use to connect to the application
# container.
readiness_probe_port = 80
# Protocol (HTTP or HTTPS) that the readiness probe should use to connect to
# the application container.
readiness_probe_protocol = "HTTP"
# Paths that should be allocated as tmpfs volumes in the Deployment container.
# Each entry in the map is a key value pair where the key is an arbitrary name
# to bind to the volume, and the value is the path in the container to mount
# the tmpfs volume.
scratch_paths = {}
# Kubernetes Secrets to be injected into the container. Each entry in the map
# represents a Secret to be injected, with the key representing the name of
# the Secret. The value is also a map, with each entry corresponding to an
# entry in the Secret, with the key corresponding to the Secret entry key and
# the value corresponding to the environment variable name.
secrets_as_env_vars = {}
# Kubernetes Secrets to be injected into the container as volume mounts. Each
# entry in the map represents a Secret to be mounted, with the key
# representing the name of the Secret and the value as a map containing
# required mountPath (file path on the container to mount the Secret to) and
# optional subPath (sub-path inside the referenced volume).
secrets_as_volumes = {}
# When true, and service_account_name is not blank, lookup and assign an
# existing ServiceAccount in the Namespace to the Pods.
service_account_exists = false
# The name of a service account to create for use with the Pods. This service
# account will be mapped to the IAM role defined in `var.iam_role_name` to
# give the pod permissions to access the AWS API. Must be unique in this
# namespace. Leave as an empty string if you do not wish to assign a Service
# Account to the Pods.
service_account_name = ""
# A list of custom Service annotations, to add to the Helm chart. See:
# https://kubernetes.io/docs/concepts/overview/working-with-objects/annotations/
service_annotations = {}
# The port to expose on the Service. This is most useful when addressing the
# Service internally to the cluster, as it is ignored when connecting from the
# Ingress resource.
service_port = 80
# Map of keys to container definitions that allow you to manage additional
# side car containers that should be included in the Pod. Note that the values
# are injected directly into the container list for the Pod Spec.
sidecar_containers = {}
# Grace period in seconds that Kubernetes will wait before terminating the
# pod. The timeout happens in parallel to preStop hook and the SIGTERM signal,
# Kubernetes does not wait for preStop to finish before beginning the grace
# period.
termination_grace_period_seconds = null
# When true, all IAM policies will be managed as dedicated policies rather
# than inline policies attached to the IAM roles. Dedicated managed policies
# are friendlier to automated policy checkers, which may scan a single
# resource for findings. As such, it is important to avoid inline policies
# when targeting compliance with various security standards.
use_managed_iam_policies = true
# A local file path where the helm chart values will be emitted. Use to debug
# issues with the helm chart values. Set to null to prevent creation of the
# file.
values_file_path = null
# When true, wait until Pods are up and healthy or wait_timeout seconds before
# exiting terraform.
wait = true
# Number of seconds to wait for Pods to become healthy before marking the
# deployment as a failure.
wait_timeout = 300
}
# ------------------------------------------------------------------------------------------------------
# DEPLOY GRUNTWORK'S K8S-SERVICE MODULE
# ------------------------------------------------------------------------------------------------------
terraform {
source = "git::git@github.com:gruntwork-io/terraform-aws-service-catalog.git//modules/services/k8s-service?ref=v0.116.1"
}
inputs = {
# ----------------------------------------------------------------------------------------------------
# REQUIRED VARIABLES
# ----------------------------------------------------------------------------------------------------
# The name of the application (e.g. my-service-stage). Used for labeling
# Kubernetes resources.
application_name = <string>
# The Docker image to run.
container_image = <object(
repository = string
tag = string
pull_policy = string
)>
# The port number on which this service's Docker container accepts incoming
# traffic.
container_port = <number>
# The number of Pods to run on the Kubernetes cluster for this service.
desired_number_of_pods = <number>
# The Kubernetes Namespace to deploy the application into.
namespace = <string>
# ----------------------------------------------------------------------------------------------------
# OPTIONAL VARIABLES
# ----------------------------------------------------------------------------------------------------
# Map of additional ports to expose for the container. The key is the name of
# the port and value contains port number and protocol.
additional_ports = null
# A list of ACM certificate ARNs to attach to the ALB. The first certificate
# in the list will be added as default certificate.
alb_acm_certificate_arns = []
# The number of consecutive health check successes required before considering
# an unhealthy target healthy.
alb_health_check_healthy_threshold = 2
# Interval between ALB health checks in seconds.
alb_health_check_interval = 30
# URL path for the endpoint that the ALB health check should ping. Defaults to
# /.
alb_health_check_path = "/"
# String value specifying the port that the ALB health check should probe. By
# default, this will be set to the traffic port (the NodePort or port where
# the service receives traffic). This can also be set to a Kubernetes named
# port, or direct integer value. See
# https://kubernetes-sigs.github.io/aws-load-balancer-controller/v2.3/guide/ingress/annotations/#healthcheck-port
# for more information.
alb_health_check_port = "traffic-port"
# Protocol (HTTP or HTTPS) that the ALB health check should use to connect to
# the application container.
alb_health_check_protocol = "HTTP"
# The HTTP status code that should be expected when doing health checks
# against the specified health check path. Accepts a single value (200),
# multiple values (200,201), or a range of values (200-300).
alb_health_check_success_codes = "200"
# The timeout, in seconds, during which no response from a target means a
# failed health check.
alb_health_check_timeout = 10
# The Docker image to use for the canary. Required if
# desired_number_of_canary_pods is greater than 0.
canary_image = null
# Allow deletion of new resources created in this upgrade when upgrade fails.
cleanup_on_fail = null
# Kubernetes ConfigMaps to be injected into the container. Each entry in the
# map represents a ConfigMap to be injected, with the key representing the
# name of the ConfigMap. The value is also a map, with each entry
# corresponding to an entry in the ConfigMap, with the key corresponding to
# the ConfigMap entry key and the value corresponding to the environment
# variable name.
configmaps_as_env_vars = {}
# Kubernetes ConfigMaps to be injected into the container as volume mounts.
# Each entry in the map represents a ConfigMap to be mounted, with the key
# representing the name of the ConfigMap and the value as a map containing
# required mountPath (file path on the container to mount the ConfigMap to)
# and optional subPath (sub-path inside the referenced volume).
configmaps_as_volumes = {}
# The protocol on which this service's Docker container accepts traffic. Must
# be one of the supported protocols:
# https://kubernetes.io/docs/concepts/services-networking/service/#protocol-support.
container_protocol = "TCP"
# The map that lets you define Kubernetes resources you want installed and
# configured as part of the chart.
custom_resources = {}
# A list of custom Deployment annotations, to add to the Helm chart. See:
# https://kubernetes.io/docs/concepts/overview/working-with-objects/annotations/
deployment_annotations = {}
# The number of canary Pods to run on the Kubernetes cluster for this service.
# If greater than 0, you must provide var.canary_image.
desired_number_of_canary_pods = 0
# The domain name for the DNS A record to bind to the Ingress resource for
# this service (e.g. service.foo.com). Depending on your external-dns
# configuration, this will also create the DNS record in the configured DNS
# service (e.g., Route53).
domain_name = null
# The TTL value of the DNS A record that is bound to the Ingress resource.
# Only used if var.domain_name is set and external-dns is deployed.
domain_propagation_ttl = null
# Configuration for using the IAM role with Service Accounts feature to
# provide permissions to the applications. This expects a map with two
# properties: `openid_connect_provider_arn` and `openid_connect_provider_url`.
# The `openid_connect_provider_arn` is the ARN of the OpenID Connect Provider
# for EKS to retrieve IAM credentials, while `openid_connect_provider_url` is
# the URL. Leave as an empty string if you do not wish to use IAM role with
# Service Accounts.
eks_iam_role_for_service_accounts_config = {"openid_connect_provider_arn":"","openid_connect_provider_url":""}
# Whether or not to enable liveness probe. Liveness checks indicate whether or
# not the container is alive. When these checks fail, the cluster will
# automatically rotate the Pod.
enable_liveness_probe = false
# Whether or not to enable readiness probe. Readiness checks indicate whether
# or not the container can accept traffic. When these checks fail, the Pods
# are automatically removed from the Service (and added back in when they
# pass).
enable_readiness_probe = false
# A map of environment variable name to environment variable value that should
# be made available to the Docker container.
env_vars = {}
# How the service will be exposed in the cluster. Must be one of `external`
# (accessible over the public Internet), `internal` (only accessible from
# within the same VPC as the cluster), `cluster-internal` (only accessible
# within the Kubernetes network), `none` (deploys as a headless service with
# no service IP).
expose_type = "cluster-internal"
# A boolean that indicates whether the access logs bucket should be destroyed,
# even if there are files in it, when you run Terraform destroy. Unless you
# are using this bucket only for test purposes, you'll want to leave this
# variable set to false.
force_destroy_ingress_access_logs = false
# The version of the k8s-service helm chart to deploy.
helm_chart_version = "v0.2.18"
# Configure the Horizontal Pod Autoscaler (HPA) information for the associated
# Deployment. HPA is disabled when this variable is set to null. Note that to
# use an HPA, you must have a corresponding service deployed to your cluster
# that exports the metrics (e.g., metrics-server
# https://github.com/kubernetes-sigs/metrics-server).
horizontal_pod_autoscaler = null
# An object defining the policy to attach to `iam_role_name` if the IAM role
# is going to be created. Accepts a map of objects, where the map keys are
# sids for IAM policy statements, and the object fields are the resources,
# actions, and the effect ("Allow" or "Deny") of the statement. Ignored if
# `iam_role_arn` is provided. Leave as null if you do not wish to use IAM role
# with Service Accounts.
iam_policy = null
# Whether or not the IAM role passed in `iam_role_name` already exists. Set to
# true if it exists, or false if it needs to be created. Defaults to false.
iam_role_exists = false
# The name of an IAM role that will be used by the pod to access the AWS API.
# If `iam_role_exists` is set to false, this role will be created. Leave as an
# empty string if you do not wish to use IAM role with Service Accounts.
iam_role_name = ""
# Set to true if the S3 bucket to store the Ingress access logs is managed
# external to this module.
ingress_access_logs_s3_bucket_already_exists = false
# The name to use for the S3 bucket where the Ingress access logs will be
# stored. If you leave this blank, a name will be generated automatically
# based on var.application_name.
ingress_access_logs_s3_bucket_name = ""
# The prefix to use for ingress access logs associated with the ALB. All logs
# will be stored in a key with this prefix. If null, the application name will
# be used.
ingress_access_logs_s3_prefix = null
# A list of custom ingress annotations, such as health checks and TLS
# certificates, to add to the Helm chart. See:
# https://kubernetes-sigs.github.io/aws-load-balancer-controller/v2.4/guide/ingress/annotations/
ingress_annotations = {}
# The protocol used by the Ingress ALB resource to communicate with the
# Service. Must be one of HTTP or HTTPS.
ingress_backend_protocol = "HTTP"
# When true, HTTP requests will automatically be redirected to use SSL
# (HTTPS). Used only when expose_type is either external or internal.
ingress_configure_ssl_redirect = true
# Assign the ingress resource to an IngressGroup. All Ingress rules of the
# group will be collapsed to a single ALB. The rules will be collapsed in
# priority order, with lower numbers being evaluated first.
ingress_group = null
# A list of maps of protocols and ports that the ALB should listen on.
ingress_listener_protocol_ports = [{"port":80,"protocol":"HTTP"},{"port":443,"protocol":"HTTPS"}]
# Path prefix that should be matched to route to the service. For Kubernetes
# Versions <1.19, Use /* to match all paths. For Kubernetes Versions >=1.19,
# use / with ingress_path_type set to Prefix to match all paths.
ingress_path = "/"
# The path type to use for the ingress rule. Refer to
# https://kubernetes.io/docs/concepts/services-networking/ingress/#path-types
# for more information.
ingress_path_type = "Prefix"
# Set to true if the Ingress SSL redirect rule is managed externally. This is
# useful when configuring Ingress grouping and you only want one service to be
# managing the SSL redirect rules. Only used if ingress_configure_ssl_redirect
# is true.
ingress_ssl_redirect_rule_already_exists = false
# Whether or not the redirect rule requires setting path type. Set to true
# when deploying to Kubernetes clusters with version >=1.19. Only used if
# ingress_configure_ssl_redirect is true.
ingress_ssl_redirect_rule_requires_path_type = true
# Controls how the ALB routes traffic to the Pods. Supports 'instance' mode
# (route traffic to NodePort and load balance across all worker nodes, relying
# on Kubernetes Service networking to route to the pods), or 'ip' mode (route
# traffic directly to the pod IP - only works with AWS VPC CNI). Must be set
# to 'ip' if using Fargate. Only used if expose_type is not cluster-internal.
ingress_target_type = "instance"
# Seconds to wait after Pod creation before liveness probe has any effect. Any
# failures during this period are ignored.
liveness_probe_grace_period_seconds = 15
# The approximate amount of time, in seconds, between liveness checks of an
# individual Target.
liveness_probe_interval_seconds = 30
# URL path for the endpoint that the liveness probe should ping.
liveness_probe_path = "/"
# Port that the liveness probe should use to connect to the application
# container.
liveness_probe_port = 80
# Protocol (HTTP or HTTPS) that the liveness probe should use to connect to
# the application container.
liveness_probe_protocol = "HTTP"
# The minimum number of pods that should be available at any given point in
# time. This is used to configure a PodDisruptionBudget for the service,
# allowing you to achieve a graceful rollout. See
# https://blog.gruntwork.io/avoiding-outages-in-your-kubernetes-cluster-using-poddisruptionbudgets-ef6a4baa5085
# for an introduction to PodDisruptionBudgets.
min_number_of_pods_available = 0
# After this number of days, Ingress log files should be transitioned from S3
# to Glacier. Set to 0 to never archive logs.
num_days_after_which_archive_ingress_log_data = 0
# After this number of days, Ingress log files should be deleted from S3. Set
# to 0 to never delete logs.
num_days_after_which_delete_ingress_log_data = 0
# Override any computed chart inputs with this map. This map is shallow merged
# to the computed chart inputs prior to passing on to the Helm Release. This
# is provided as a workaround while the terraform module does not support a
# particular input value that is exposed in the underlying chart. Please
# always file a GitHub issue to request exposing additional underlying input
# values prior to using this variable.
override_chart_inputs = {}
# A list of custom Pod annotations, to add to the Helm chart. See:
# https://kubernetes.io/docs/concepts/overview/working-with-objects/annotations/
pod_annotations = {}
# Seconds to wait after Pod creation before liveness probe has any effect. Any
# failures during this period are ignored.
readiness_probe_grace_period_seconds = 15
# The approximate amount of time, in seconds, between liveness checks of an
# individual Target.
readiness_probe_interval_seconds = 30
# URL path for the endpoint that the readiness probe should ping.
readiness_probe_path = "/"
# Port that the readiness probe should use to connect to the application
# container.
readiness_probe_port = 80
# Protocol (HTTP or HTTPS) that the readiness probe should use to connect to
# the application container.
readiness_probe_protocol = "HTTP"
# Paths that should be allocated as tmpfs volumes in the Deployment container.
# Each entry in the map is a key value pair where the key is an arbitrary name
# to bind to the volume, and the value is the path in the container to mount
# the tmpfs volume.
scratch_paths = {}
# Kubernetes Secrets to be injected into the container. Each entry in the map
# represents a Secret to be injected, with the key representing the name of
# the Secret. The value is also a map, with each entry corresponding to an
# entry in the Secret, with the key corresponding to the Secret entry key and
# the value corresponding to the environment variable name.
secrets_as_env_vars = {}
# Kubernetes Secrets to be injected into the container as volume mounts. Each
# entry in the map represents a Secret to be mounted, with the key
# representing the name of the Secret and the value as a map containing
# required mountPath (file path on the container to mount the Secret to) and
# optional subPath (sub-path inside the referenced volume).
secrets_as_volumes = {}
# When true, and service_account_name is not blank, lookup and assign an
# existing ServiceAccount in the Namespace to the Pods.
service_account_exists = false
# The name of a service account to create for use with the Pods. This service
# account will be mapped to the IAM role defined in `var.iam_role_name` to
# give the pod permissions to access the AWS API. Must be unique in this
# namespace. Leave as an empty string if you do not wish to assign a Service
# Account to the Pods.
service_account_name = ""
# A list of custom Service annotations, to add to the Helm chart. See:
# https://kubernetes.io/docs/concepts/overview/working-with-objects/annotations/
service_annotations = {}
# The port to expose on the Service. This is most useful when addressing the
# Service internally to the cluster, as it is ignored when connecting from the
# Ingress resource.
service_port = 80
# Map of keys to container definitions that allow you to manage additional
# side car containers that should be included in the Pod. Note that the values
# are injected directly into the container list for the Pod Spec.
sidecar_containers = {}
# Grace period in seconds that Kubernetes will wait before terminating the
# pod. The timeout happens in parallel to preStop hook and the SIGTERM signal,
# Kubernetes does not wait for preStop to finish before beginning the grace
# period.
termination_grace_period_seconds = null
# When true, all IAM policies will be managed as dedicated policies rather
# than inline policies attached to the IAM roles. Dedicated managed policies
# are friendlier to automated policy checkers, which may scan a single
# resource for findings. As such, it is important to avoid inline policies
# when targeting compliance with various security standards.
use_managed_iam_policies = true
# A local file path where the helm chart values will be emitted. Use to debug
# issues with the helm chart values. Set to null to prevent creation of the
# file.
values_file_path = null
# When true, wait until Pods are up and healthy or wait_timeout seconds before
# exiting terraform.
wait = true
# Number of seconds to wait for Pods to become healthy before marking the
# deployment as a failure.
wait_timeout = 300
}
Reference
- Inputs
- Outputs
Required
application_name
stringThe name of the application (e.g. my-service-stage). Used for labeling Kubernetes resources.
container_image
object(…)The Docker image to run.
object({
# Repository of the docker image (e.g. gruntwork/frontend-service)
repository = string
# The tag of the docker image to deploy.
tag = string
# The image pull policy. Can be one of IfNotPresent, Always, or Never.
pull_policy = string
})
container_port
numberThe port number on which this service's Docker container accepts incoming traffic.
desired_number_of_pods
numberThe number of Pods to run on the Kubernetes cluster for this service.
namespace
stringThe Kubernetes Namespace to deploy the application into.
Optional
additional_ports
map(object(…))Map of additional ports to expose for the container. The key is the name of the port and value contains port number and protocol.
map(object({
port : number
protocol : string
}))
null
Example
additional_ports = {
prometheus = {
port = 9102
protocol = "TCP"
}
}
alb_acm_certificate_arns
list(string)A list of ACM certificate ARNs to attach to the ALB. The first certificate in the list will be added as default certificate.
[]
The number of consecutive health check successes required before considering an unhealthy target healthy.
2
Interval between ALB health checks in seconds.
30
alb_health_check_path
stringURL path for the endpoint that the ALB health check should ping. Defaults to /.
"/"
alb_health_check_port
stringString value specifying the port that the ALB health check should probe. By default, this will be set to the traffic port (the NodePort or port where the service receives traffic). This can also be set to a Kubernetes named port, or direct integer value. See https://kubernetes-sigs.github.io/aws-load-balancer-controller/v2.3/guide/ingress/annotations/#healthcheck-port for more information.
"traffic-port"
Protocol (HTTP or HTTPS) that the ALB health check should use to connect to the application container.
"HTTP"
The HTTP status code that should be expected when doing health checks against the specified health check path. Accepts a single value (200), multiple values (200,201), or a range of values (200-300).
"200"
alb_health_check_timeout
numberThe timeout, in seconds, during which no response from a target means a failed health check.
10
canary_image
object(…)The Docker image to use for the canary. Required if desired_number_of_canary_pods is greater than 0.
object({
# Repository of the docker image (e.g. gruntwork/frontend-service)
repository = string
# The tag of the docker image to deploy.
tag = string
# The image pull policy. Can be one of IfNotPresent, Always, or Never.
pull_policy = string
})
null
cleanup_on_fail
boolAllow deletion of new resources created in this upgrade when upgrade fails.
null
configmaps_as_env_vars
map(map(…))Kubernetes ConfigMaps to be injected into the container. Each entry in the map represents a ConfigMap to be injected, with the key representing the name of the ConfigMap. The value is also a map, with each entry corresponding to an entry in the ConfigMap, with the key corresponding to the ConfigMap entry key and the value corresponding to the environment variable name.
map(map(string))
{}
Example
Example: This will inject the foo key of the ConfigMap myconfig as the environment variable MY_CONFIG.
{
myconfig = {
foo = "MY_CONFIG"
}
}
configmaps_as_volumes
map(any)Kubernetes ConfigMaps to be injected into the container as volume mounts. Each entry in the map represents a ConfigMap to be mounted, with the key representing the name of the ConfigMap and the value as a map containing required mountPath (file path on the container to mount the ConfigMap to) and optional subPath (sub-path inside the referenced volume).
Any types represent complex values of variable type. For details, please consult `variables.tf` in the source repo.
{}
Example
Example: This will mount the ConfigMap myconfig to the path /etc/myconfig
{
myconfig = {
mount_path = "/etc/myconfig"
}
}
Example: This will mount the ConfigMap myconfig to the path /etc/nginx/nginx.conf
{
myconfig = {
mount_path = "/etc/nginx/nginx.conf"
sub_path = "nginx.conf"
}
}
container_protocol
stringThe protocol on which this service's Docker container accepts traffic. Must be one of the supported protocols: https://kubernetes.io/docs/concepts/services-networking/service/#protocol-support.
"TCP"
custom_resources
map(string)The map that lets you define Kubernetes resources you want installed and configured as part of the chart.
{}
Example
Example: the following example creates a custom ConfigMap from a string and a Secret from a file.
{
custom_configmap = <<EOF
apiVersion: v1
kind: ConfigMap
metadata:
name: example
stringData:
key: value
EOF
custom_secret = file("${path.module}/secret.yaml")
}
deployment_annotations
map(string)A list of custom Deployment annotations, to add to the Helm chart. See: https://kubernetes.io/docs/concepts/overview/working-with-objects/annotations/
{}
Example
{
"prometheus.io/scrape" : "true"
}
The number of canary Pods to run on the Kubernetes cluster for this service. If greater than 0, you must provide canary_image
.
0
domain_name
stringThe domain name for the DNS A record to bind to the Ingress resource for this service (e.g. service.foo.com). Depending on your external-dns configuration, this will also create the DNS record in the configured DNS service (e.g., Route53).
null
domain_propagation_ttl
numberThe TTL value of the DNS A record that is bound to the Ingress resource. Only used if domain_name
is set and external-dns is deployed.
null
Configuration for using the IAM role with Service Accounts feature to provide permissions to the applications. This expects a map with two properties: openid_connect_provider_arn
and openid_connect_provider_url
. The openid_connect_provider_arn
is the ARN of the OpenID Connect Provider for EKS to retrieve IAM credentials, while openid_connect_provider_url
is the URL. Leave as an empty string if you do not wish to use IAM role with Service Accounts.
object({
openid_connect_provider_arn = string
openid_connect_provider_url = string
})
{
openid_connect_provider_arn = "",
openid_connect_provider_url = ""
}
Whether or not to enable liveness probe. Liveness checks indicate whether or not the container is alive. When these checks fail, the cluster will automatically rotate the Pod.
false
Whether or not to enable readiness probe. Readiness checks indicate whether or not the container can accept traffic. When these checks fail, the Pods are automatically removed from the Service (and added back in when they pass).
false
env_vars
map(string)A map of environment variable name to environment variable value that should be made available to the Docker container.
{}
expose_type
stringHow the service will be exposed in the cluster. Must be one of external
(accessible over the public Internet), internal
(only accessible from within the same VPC as the cluster), cluster-internal
(only accessible within the Kubernetes network), none
(deploys as a headless service with no service IP).
"cluster-internal"
A boolean that indicates whether the access logs bucket should be destroyed, even if there are files in it, when you run Terraform destroy. Unless you are using this bucket only for test purposes, you'll want to leave this variable set to false.
false
helm_chart_version
stringThe version of the k8s-service helm chart to deploy.
"v0.2.18"
horizontal_pod_autoscaler
object(…)Configure the Horizontal Pod Autoscaler (HPA) information for the associated Deployment. HPA is disabled when this variable is set to null. Note that to use an HPA, you must have a corresponding service deployed to your cluster that exports the metrics (e.g., metrics-server https://github.com/kubernetes-sigs/metrics-server).
object({
# The minimum amount of replicas allowed
min_replicas = number
# The maximum amount of replicas allowed
max_replicas = number
# The target average CPU utilization (as a percentage) to be used with the metrics. E.g., setting this to 60 means
# that the HPA controller will keep the average utilization of the CPU in Pods in the scaling target at 60%.
avg_cpu_utilization = number
# The target average Memory utilization (as a percentage) to be used with the metrics. Works the same as
# avg_cpu_utilization.
avg_mem_utilization = number
})
null
iam_policy
map(object(…))An object defining the policy to attach to iam_role_name
if the IAM role is going to be created. Accepts a map of objects, where the map keys are sids for IAM policy statements, and the object fields are the resources, actions, and the effect ('Allow' or 'Deny') of the statement. Ignored if iam_role_arn
is provided. Leave as null if you do not wish to use IAM role with Service Accounts.
map(object({
resources = list(string)
actions = list(string)
effect = string
}))
null
Example
iam_policy = {
S3Access = {
actions = ["s3:*"]
resources = ["arn:aws:s3:::mybucket"]
effect = "Allow"
},
SecretsManagerAccess = {
actions = ["secretsmanager:GetSecretValue"],
resources = ["arn:aws:secretsmanager:us-east-1:0123456789012:secret:mysecert"]
effect = "Allow"
}
}
iam_role_exists
boolWhether or not the IAM role passed in iam_role_name
already exists. Set to true if it exists, or false if it needs to be created. Defaults to false.
false
iam_role_name
stringThe name of an IAM role that will be used by the pod to access the AWS API. If iam_role_exists
is set to false, this role will be created. Leave as an empty string if you do not wish to use IAM role with Service Accounts.
""
Set to true if the S3 bucket to store the Ingress access logs is managed external to this module.
false
The name to use for the S3 bucket where the Ingress access logs will be stored. If you leave this blank, a name will be generated automatically based on application_name
.
""
The prefix to use for ingress access logs associated with the ALB. All logs will be stored in a key with this prefix. If null, the application name will be used.
null
ingress_annotations
map(string)A list of custom ingress annotations, such as health checks and TLS certificates, to add to the Helm chart. See: https://kubernetes-sigs.github.io/aws-load-balancer-controller/v2.4/guide/ingress/annotations/
{}
Example
{
"alb.ingress.kubernetes.io/shield-advanced-protection" : "true"
}
ingress_backend_protocol
stringThe protocol used by the Ingress ALB resource to communicate with the Service. Must be one of HTTP or HTTPS.
"HTTP"
When true, HTTP requests will automatically be redirected to use SSL (HTTPS). Used only when expose_type is either external or internal.
true
ingress_group
object(…)Assign the ingress resource to an IngressGroup. All Ingress rules of the group will be collapsed to a single ALB. The rules will be collapsed in priority order, with lower numbers being evaluated first.
object({
# Ingress group to assign to.
name = string
# The priority of the rules in this Ingress. Smaller numbers have higher priority.
priority = number
})
null
ingress_listener_protocol_ports
list(object(…))A list of maps of protocols and ports that the ALB should listen on.
list(object({
protocol = string
port = number
}))
[
{
port = 80,
protocol = "HTTP"
},
{
port = 443,
protocol = "HTTPS"
}
]
ingress_path
stringPath prefix that should be matched to route to the service. For Kubernetes Versions <1.19, Use /* to match all paths. For Kubernetes Versions >=1.19, use / with ingress_path_type set to Prefix to match all paths.
"/"
ingress_path_type
stringThe path type to use for the ingress rule. Refer to https://kubernetes.io/docs/concepts/services-networking/ingress/#path-types for more information.
"Prefix"
Set to true if the Ingress SSL redirect rule is managed externally. This is useful when configuring Ingress grouping and you only want one service to be managing the SSL redirect rules. Only used if ingress_configure_ssl_redirect is true.
false
Whether or not the redirect rule requires setting path type. Set to true when deploying to Kubernetes clusters with version >=1.19. Only used if ingress_configure_ssl_redirect is true.
true
ingress_target_type
stringControls how the ALB routes traffic to the Pods. Supports 'instance' mode (route traffic to NodePort and load balance across all worker nodes, relying on Kubernetes Service networking to route to the pods), or 'ip' mode (route traffic directly to the pod IP - only works with AWS VPC CNI). Must be set to 'ip' if using Fargate. Only used if expose_type is not cluster-internal.
"instance"
Seconds to wait after Pod creation before liveness probe has any effect. Any failures during this period are ignored.
15
The approximate amount of time, in seconds, between liveness checks of an individual Target.
30
liveness_probe_path
stringURL path for the endpoint that the liveness probe should ping.
"/"
liveness_probe_port
numberPort that the liveness probe should use to connect to the application container.
80
liveness_probe_protocol
stringProtocol (HTTP or HTTPS) that the liveness probe should use to connect to the application container.
"HTTP"
The minimum number of pods that should be available at any given point in time. This is used to configure a PodDisruptionBudget for the service, allowing you to achieve a graceful rollout. See https://blog.gruntwork.io/avoiding-outages-in-your-kubernetes-cluster-using-poddisruptionbudgets-ef6a4baa5085 for an introduction to PodDisruptionBudgets.
0
After this number of days, Ingress log files should be transitioned from S3 to Glacier. Set to 0 to never archive logs.
0
After this number of days, Ingress log files should be deleted from S3. Set to 0 to never delete logs.
0
Override any computed chart inputs with this map. This map is shallow merged to the computed chart inputs prior to passing on to the Helm Release. This is provided as a workaround while the terraform module does not support a particular input value that is exposed in the underlying chart. Please always file a GitHub issue to request exposing additional underlying input values prior to using this variable.
Any types represent complex values of variable type. For details, please consult `variables.tf` in the source repo.
{}
Details
Ideally we would define a concrete type here, but since the input value spec for the chart has dynamic optional
values, we can't use a concrete object type for Terraform. Also, setting a type spec here will defeat the purpose of
the escape hatch since it requires defining new input values here before users can use it.
pod_annotations
map(string)A list of custom Pod annotations, to add to the Helm chart. See: https://kubernetes.io/docs/concepts/overview/working-with-objects/annotations/
{}
Example
{
"prometheus.io/scrape" : "true"
}
Seconds to wait after Pod creation before liveness probe has any effect. Any failures during this period are ignored.
15
The approximate amount of time, in seconds, between liveness checks of an individual Target.
30
readiness_probe_path
stringURL path for the endpoint that the readiness probe should ping.
"/"
readiness_probe_port
numberPort that the readiness probe should use to connect to the application container.
80
readiness_probe_protocol
stringProtocol (HTTP or HTTPS) that the readiness probe should use to connect to the application container.
"HTTP"
scratch_paths
map(string)Paths that should be allocated as tmpfs volumes in the Deployment container. Each entry in the map is a key value pair where the key is an arbitrary name to bind to the volume, and the value is the path in the container to mount the tmpfs volume.
{}
Example
Example: This will mount the tmpfs volume "foo" to the path "/mnt/scratch"
{
foo = "/mnt/scratch"
}
secrets_as_env_vars
map(map(…))Kubernetes Secrets to be injected into the container. Each entry in the map represents a Secret to be injected, with the key representing the name of the Secret. The value is also a map, with each entry corresponding to an entry in the Secret, with the key corresponding to the Secret entry key and the value corresponding to the environment variable name.
map(map(string))
{}
Example
Example: This will inject the foo key of the Secret mysecret as the environment variable MY_SECRET.
{
mysecret = {
foo = "MY_SECRET"
}
}
secrets_as_volumes
map(any)Kubernetes Secrets to be injected into the container as volume mounts. Each entry in the map represents a Secret to be mounted, with the key representing the name of the Secret and the value as a map containing required mountPath (file path on the container to mount the Secret to) and optional subPath (sub-path inside the referenced volume).
Any types represent complex values of variable type. For details, please consult `variables.tf` in the source repo.
{}
Example
Example: This will mount the Secret mysecret to the path /etc/mysecret
{
mysecret = {
mount_path = "/etc/mysecret"
}
}
Example: This will mount the Secret mysecret to the path /etc/nginx/nginx.conf
{
mysecret = {
mount_path = "/etc/nginx/nginx.conf"
sub_path = "nginx.conf"
}
}
When true, and service_account_name is not blank, lookup and assign an existing ServiceAccount in the Namespace to the Pods.
false
service_account_name
stringThe name of a service account to create for use with the Pods. This service account will be mapped to the IAM role defined in <a href="#iam_role_name"><code>iam_role_name</code></a>
to give the pod permissions to access the AWS API. Must be unique in this namespace. Leave as an empty string if you do not wish to assign a Service Account to the Pods.
""
service_annotations
map(string)A list of custom Service annotations, to add to the Helm chart. See: https://kubernetes.io/docs/concepts/overview/working-with-objects/annotations/
{}
Example
{
"prometheus.io/scrape" : "true"
}
service_port
numberThe port to expose on the Service. This is most useful when addressing the Service internally to the cluster, as it is ignored when connecting from the Ingress resource.
80
Map of keys to container definitions that allow you to manage additional side car containers that should be included in the Pod. Note that the values are injected directly into the container list for the Pod Spec.
Any types represent complex values of variable type. For details, please consult `variables.tf` in the source repo.
{}
Example
sidecar_containers = {
datadog = {
image = "datadog/agent:latest"
env = [
{
name = "DD_API_KEY"
value = "ASDF-1234"
},
{
name = "SD_BACKEND"
value = "docker"
},
]
}
}
Details
Ideally we would define a concrete type here, but since the container spec for Pods have dynamic optional values, we
can't use a concrete object type for Terraform.
Grace period in seconds that Kubernetes will wait before terminating the pod. The timeout happens in parallel to preStop hook and the SIGTERM signal, Kubernetes does not wait for preStop to finish before beginning the grace period.
null
When true, all IAM policies will be managed as dedicated policies rather than inline policies attached to the IAM roles. Dedicated managed policies are friendlier to automated policy checkers, which may scan a single resource for findings. As such, it is important to avoid inline policies when targeting compliance with various security standards.
true
values_file_path
stringA local file path where the helm chart values will be emitted. Use to debug issues with the helm chart values. Set to null to prevent creation of the file.
null
wait
boolWhen true, wait until Pods are up and healthy or wait_timeout seconds before exiting terraform.
true
wait_timeout
numberNumber of seconds to wait for Pods to become healthy before marking the deployment as a failure.
300