EKS Karpenter
Overview
This service contains Terraform code to deploy Karpenter to Elastic Kubernetes Service(EKS).
From the Karpenter Project:
Karpenter automatically launches just the right compute resources to handle your cluster's applications. It is designed to let you take full advantage of the cloud with fast and simple compute provisioning for Kubernetes clusters.
Features
- Creates the required resources to deploy Karpenter to EKS
Learn
This repo is a part of the Gruntwork Service Catalog, a collection of reusable, battle-tested, production ready infrastructure code. If you’ve never used the Service Catalog before, make sure to read How to use the Gruntwork Service Catalog!
Under the hood, this is all implemented using Terraform modules from the Gruntwork terraform-aws-eks repo. If you are a subscriber and don’t have access to this repo, email support@gruntwork.io.
Core concepts
For detailed information on how Karpenter is deployed to EKS, see the documentation in the terraform-aws-eks repo.
Repo organization
- modules: the main implementation code for this repo, broken down into multiple standalone, orthogonal submodules.
- examples: This folder contains working examples of how to use the submodules.
- test: Automated tests for the modules and examples.
Deploy
Non-production deployment (quick start for learning)
If you just want to try this repo out for experimenting and learning, check out the following resources:
- examples/for-learning-and-testing folder: The
examples/for-learning-and-testing
folder contains standalone sample code optimized for learning, experimenting, and testing (but not direct production usage).
Production deployment
If you want to deploy this repo in production, check out the following resources:
-
examples/for-production folder: The
examples/for-production
folder contains sample code optimized for direct usage in production. This is code from the Gruntwork Reference Architecture, and it shows you how we build an end-to-end, integrated tech stack on top of the Gruntwork Service Catalog. -
How to deploy a production-grade Kubernetes cluster on AWS: A step-by-step guide for deploying a production-grade EKS cluster on AWS using the code in this repo.
Sample Usage
- Terraform
- Terragrunt
# ------------------------------------------------------------------------------------------------------
# DEPLOY GRUNTWORK'S EKS-KARPENTER MODULE
# ------------------------------------------------------------------------------------------------------
module "eks_karpenter" {
source = "git::git@github.com:gruntwork-io/terraform-aws-service-catalog.git//modules/services/eks-karpenter?ref=v0.118.2"
# ----------------------------------------------------------------------------------------------------
# REQUIRED VARIABLES
# ----------------------------------------------------------------------------------------------------
# The AWS region in which all resources will be created
aws_region = <string>
# URL endpoint of the Kubernetes control plane provided by EKS.
eks_cluster_endpoint = <string>
# The name of the EKS cluster where the core services will be deployed into.
eks_cluster_name = <string>
# The ARN of the EKS OIDC provider. This is required if creating IRSA for the
# Karpenter Controller.
eks_openid_connect_provider_arn = <string>
# The URL of the EKS OIDC provider. This is required if creating IRSA for the
# Karpenter Controller.
eks_openid_connect_provider_url = <string>
# ----------------------------------------------------------------------------------------------------
# OPTIONAL VARIABLES
# ----------------------------------------------------------------------------------------------------
# Optionally create an IAM Role for Service Account (IRSA) for the Karpenter
# Controller.
create_karpenter_controller_irsa = true
# Conditional flag to optionally create the Karpenter Deprovisioning Queue.
create_karpenter_deprovisioning_queue = true
# Conditional flag to create the Karpenter Node IAM Role. If this is set to
# false, then an existing IAM Role must be provided with the
# `karpenter_node_iam_role_arn` variable
create_karpenter_node_iam_role = true
# Conditional flag to optionally create resources in this module.
create_resources = true
# Tags to apply to all AWS resources managed by this module.
default_tags = {}
# Additional Helm chart values to pass to the Karpenter Helm chart. See the
# official Karpenter Helm chart values file and documentation for available
# configuration options.
karpenter_chart_additional_values = {}
# The Helm chart name for the Karpenter chart.
karpenter_chart_name = "karpenter"
# The k8s namespace that the Karpenter Helm chart will be deployed to.
karpenter_chart_namespace = "karpenter"
# The Helm release name for the Karpenter chart.
karpenter_chart_release_name = "karpenter"
# The Helm repository to obtain the Karpenter chart from.
karpenter_chart_repository = "oci://public.ecr.aws/karpenter"
# Whether or not to install CRDs with the Karpenter Helm Chart. This should be
# set to true if using the karpenter-crd Helm Chart.
karpenter_chart_skip_crds = false
# The version of the Karpenter Helm chart.
karpenter_chart_version = "v0.32.7"
# Provide an existing IAM Role ARN to be used with the Karpenter Controller
# Service Account. This is required if `create_karpenter_controller_irsa` is
# set to false.
karpenter_controller_existing_role_arn = null
# A map of custom tags to apply to the Karpenter Controller IAM Policies if
# enabled. The key is the tag name and the value is the tag value.
karpenter_controller_iam_policy_tags = {}
# Additional tags to add to the Karpenter Controller IAM Role.
karpenter_controller_iam_role_tags = {}
# The Helm chart name for the Karpenter CRD chart.
karpenter_crd_chart_name = "karpenter-crd"
# The k8s namespace that the Karpenter CRD Helm chart will be deployed to.
karpenter_crd_chart_namespace = "karpenter"
# The Helm release name for the Karpenter CRD chart.
karpenter_crd_chart_release_name = "karpenter-crd"
# The Helm repository to obtain the Karpenter CRD chart from.
karpenter_crd_chart_repository = "oci://public.ecr.aws/karpenter"
# The version of the Karpenter CRD Helm chart. This should typically be the
# same version as karpenter_chart_version.
karpenter_crd_chart_version = "v0.32.7"
# Whether or not to create the Karpneter CRDs via the karpenter-crd Helm
# chart. It is suggested to manage the Karpenter CRDs via this Helm chart.
karpenter_crd_helm_create = true
# A map of custom tags to apply to the Karpenter Deprovisioning Queue IAM
# Policies if enabled. The key is the tag name and the value is the tag value.
karpenter_deprovisioning_queue_iam_policy_tags = {}
# Additional tags to add to the Karpenter Deprovisioning Queue.
karpenter_deprovisioning_queue_tags = {}
# A tag that is used by Karpenter to discover resources.
karpenter_discovery_tag = "karpenter.sh/discovery"
# ARN of the policy that is used to set the permissions boundary for the role.
karpenter_irsa_permissions_boundary = null
# Use an existing IAM Role to be used for the Karpenter Node Instance Profile.
# This is required if `create_karpenter_node_iam_role` is set to false. This
# should be the ARN of the IAM Role.
karpenter_node_existing_iam_role_arn = null
# Use an existing IAM Role to be used for the Karpenter Node Instance Profile.
# This is required if `create_karpenter_node_iam_role` is set to false. This
# should be the Name of the IAM Role.
karpenter_node_existing_iam_role_name = null
# A description of the Karpenter Node IAM Role.
karpenter_node_iam_role_description = "IAM Role attached to nodes launched by Karpenter."
# Maximum session duration (in seconds) that you want to set for the Karpenter
# Node role. Value can be between 3600 and 43200.
karpenter_node_iam_role_max_session_duration = 3600
# Optionally provide a name for the Karpenter Node IAM Role. If unset, a name
# will be generated.
karpenter_node_iam_role_name = null
# Optionally provide a path to the Karpenter Node IAM Role.
karpenter_node_iam_role_path = null
# ARN of the policy that is used to set the permissions boundary for the role.
karpenter_node_iam_role_permissions_boundary = null
# Additional tags to add to the Karpenter Node IAM Role.
karpenter_node_iam_role_tags = {}
# Optionally use an IAM name prefix for the Karpenter IAM Role.
karpenter_node_iam_role_use_name_prefix = false
# Optionally provide a name for the Karpenter service account that will be
# associated with IRSA.
karpenter_service_account_name = "karpenter"
}
# ------------------------------------------------------------------------------------------------------
# DEPLOY GRUNTWORK'S EKS-KARPENTER MODULE
# ------------------------------------------------------------------------------------------------------
terraform {
source = "git::git@github.com:gruntwork-io/terraform-aws-service-catalog.git//modules/services/eks-karpenter?ref=v0.118.2"
}
inputs = {
# ----------------------------------------------------------------------------------------------------
# REQUIRED VARIABLES
# ----------------------------------------------------------------------------------------------------
# The AWS region in which all resources will be created
aws_region = <string>
# URL endpoint of the Kubernetes control plane provided by EKS.
eks_cluster_endpoint = <string>
# The name of the EKS cluster where the core services will be deployed into.
eks_cluster_name = <string>
# The ARN of the EKS OIDC provider. This is required if creating IRSA for the
# Karpenter Controller.
eks_openid_connect_provider_arn = <string>
# The URL of the EKS OIDC provider. This is required if creating IRSA for the
# Karpenter Controller.
eks_openid_connect_provider_url = <string>
# ----------------------------------------------------------------------------------------------------
# OPTIONAL VARIABLES
# ----------------------------------------------------------------------------------------------------
# Optionally create an IAM Role for Service Account (IRSA) for the Karpenter
# Controller.
create_karpenter_controller_irsa = true
# Conditional flag to optionally create the Karpenter Deprovisioning Queue.
create_karpenter_deprovisioning_queue = true
# Conditional flag to create the Karpenter Node IAM Role. If this is set to
# false, then an existing IAM Role must be provided with the
# `karpenter_node_iam_role_arn` variable
create_karpenter_node_iam_role = true
# Conditional flag to optionally create resources in this module.
create_resources = true
# Tags to apply to all AWS resources managed by this module.
default_tags = {}
# Additional Helm chart values to pass to the Karpenter Helm chart. See the
# official Karpenter Helm chart values file and documentation for available
# configuration options.
karpenter_chart_additional_values = {}
# The Helm chart name for the Karpenter chart.
karpenter_chart_name = "karpenter"
# The k8s namespace that the Karpenter Helm chart will be deployed to.
karpenter_chart_namespace = "karpenter"
# The Helm release name for the Karpenter chart.
karpenter_chart_release_name = "karpenter"
# The Helm repository to obtain the Karpenter chart from.
karpenter_chart_repository = "oci://public.ecr.aws/karpenter"
# Whether or not to install CRDs with the Karpenter Helm Chart. This should be
# set to true if using the karpenter-crd Helm Chart.
karpenter_chart_skip_crds = false
# The version of the Karpenter Helm chart.
karpenter_chart_version = "v0.32.7"
# Provide an existing IAM Role ARN to be used with the Karpenter Controller
# Service Account. This is required if `create_karpenter_controller_irsa` is
# set to false.
karpenter_controller_existing_role_arn = null
# A map of custom tags to apply to the Karpenter Controller IAM Policies if
# enabled. The key is the tag name and the value is the tag value.
karpenter_controller_iam_policy_tags = {}
# Additional tags to add to the Karpenter Controller IAM Role.
karpenter_controller_iam_role_tags = {}
# The Helm chart name for the Karpenter CRD chart.
karpenter_crd_chart_name = "karpenter-crd"
# The k8s namespace that the Karpenter CRD Helm chart will be deployed to.
karpenter_crd_chart_namespace = "karpenter"
# The Helm release name for the Karpenter CRD chart.
karpenter_crd_chart_release_name = "karpenter-crd"
# The Helm repository to obtain the Karpenter CRD chart from.
karpenter_crd_chart_repository = "oci://public.ecr.aws/karpenter"
# The version of the Karpenter CRD Helm chart. This should typically be the
# same version as karpenter_chart_version.
karpenter_crd_chart_version = "v0.32.7"
# Whether or not to create the Karpneter CRDs via the karpenter-crd Helm
# chart. It is suggested to manage the Karpenter CRDs via this Helm chart.
karpenter_crd_helm_create = true
# A map of custom tags to apply to the Karpenter Deprovisioning Queue IAM
# Policies if enabled. The key is the tag name and the value is the tag value.
karpenter_deprovisioning_queue_iam_policy_tags = {}
# Additional tags to add to the Karpenter Deprovisioning Queue.
karpenter_deprovisioning_queue_tags = {}
# A tag that is used by Karpenter to discover resources.
karpenter_discovery_tag = "karpenter.sh/discovery"
# ARN of the policy that is used to set the permissions boundary for the role.
karpenter_irsa_permissions_boundary = null
# Use an existing IAM Role to be used for the Karpenter Node Instance Profile.
# This is required if `create_karpenter_node_iam_role` is set to false. This
# should be the ARN of the IAM Role.
karpenter_node_existing_iam_role_arn = null
# Use an existing IAM Role to be used for the Karpenter Node Instance Profile.
# This is required if `create_karpenter_node_iam_role` is set to false. This
# should be the Name of the IAM Role.
karpenter_node_existing_iam_role_name = null
# A description of the Karpenter Node IAM Role.
karpenter_node_iam_role_description = "IAM Role attached to nodes launched by Karpenter."
# Maximum session duration (in seconds) that you want to set for the Karpenter
# Node role. Value can be between 3600 and 43200.
karpenter_node_iam_role_max_session_duration = 3600
# Optionally provide a name for the Karpenter Node IAM Role. If unset, a name
# will be generated.
karpenter_node_iam_role_name = null
# Optionally provide a path to the Karpenter Node IAM Role.
karpenter_node_iam_role_path = null
# ARN of the policy that is used to set the permissions boundary for the role.
karpenter_node_iam_role_permissions_boundary = null
# Additional tags to add to the Karpenter Node IAM Role.
karpenter_node_iam_role_tags = {}
# Optionally use an IAM name prefix for the Karpenter IAM Role.
karpenter_node_iam_role_use_name_prefix = false
# Optionally provide a name for the Karpenter service account that will be
# associated with IRSA.
karpenter_service_account_name = "karpenter"
}
Reference
- Inputs
- Outputs
Required
aws_region
stringThe AWS region in which all resources will be created
eks_cluster_endpoint
stringURL endpoint of the Kubernetes control plane provided by EKS.
eks_cluster_name
stringThe name of the EKS cluster where the core services will be deployed into.
The ARN of the EKS OIDC provider. This is required if creating IRSA for the Karpenter Controller.
The URL of the EKS OIDC provider. This is required if creating IRSA for the Karpenter Controller.
Optional
Optionally create an IAM Role for Service Account (IRSA) for the Karpenter Controller.
true
Conditional flag to optionally create the Karpenter Deprovisioning Queue.
true
Conditional flag to create the Karpenter Node IAM Role. If this is set to false, then an existing IAM Role must be provided with the karpenter_node_iam_role_arn
variable
true
create_resources
boolConditional flag to optionally create resources in this module.
true
default_tags
map(string)Tags to apply to all AWS resources managed by this module.
{}
Additional Helm chart values to pass to the Karpenter Helm chart. See the official Karpenter Helm chart values file and documentation for available configuration options.
Any types represent complex values of variable type. For details, please consult `variables.tf` in the source repo.
{}
karpenter_chart_name
stringThe Helm chart name for the Karpenter chart.
"karpenter"
The k8s namespace that the Karpenter Helm chart will be deployed to.
"karpenter"
The Helm release name for the Karpenter chart.
"karpenter"
The Helm repository to obtain the Karpenter chart from.
"oci://public.ecr.aws/karpenter"
Whether or not to install CRDs with the Karpenter Helm Chart. This should be set to true if using the karpenter-crd Helm Chart.
false
karpenter_chart_version
stringThe version of the Karpenter Helm chart.
"v0.32.7"
Provide an existing IAM Role ARN to be used with the Karpenter Controller Service Account. This is required if create_karpenter_controller_irsa
is set to false.
null
karpenter_controller_iam_policy_tags
map(string)A map of custom tags to apply to the Karpenter Controller IAM Policies if enabled. The key is the tag name and the value is the tag value.
{}
karpenter_controller_iam_role_tags
map(string)Additional tags to add to the Karpenter Controller IAM Role.
{}
karpenter_crd_chart_name
stringThe Helm chart name for the Karpenter CRD chart.
"karpenter-crd"
The k8s namespace that the Karpenter CRD Helm chart will be deployed to.
"karpenter"
The Helm release name for the Karpenter CRD chart.
"karpenter-crd"
The Helm repository to obtain the Karpenter CRD chart from.
"oci://public.ecr.aws/karpenter"
The version of the Karpenter CRD Helm chart. This should typically be the same version as karpenter_chart_version.
"v0.32.7"
Whether or not to create the Karpneter CRDs via the karpenter-crd Helm chart. It is suggested to manage the Karpenter CRDs via this Helm chart.
true
A map of custom tags to apply to the Karpenter Deprovisioning Queue IAM Policies if enabled. The key is the tag name and the value is the tag value.
{}
karpenter_deprovisioning_queue_tags
map(string)Additional tags to add to the Karpenter Deprovisioning Queue.
{}
karpenter_discovery_tag
stringA tag that is used by Karpenter to discover resources.
"karpenter.sh/discovery"
ARN of the policy that is used to set the permissions boundary for the role.
null
Use an existing IAM Role to be used for the Karpenter Node Instance Profile. This is required if create_karpenter_node_iam_role
is set to false. This should be the ARN of the IAM Role.
null
Use an existing IAM Role to be used for the Karpenter Node Instance Profile. This is required if create_karpenter_node_iam_role
is set to false. This should be the Name of the IAM Role.
null
A description of the Karpenter Node IAM Role.
"IAM Role attached to nodes launched by Karpenter."
Maximum session duration (in seconds) that you want to set for the Karpenter Node role. Value can be between 3600 and 43200.
3600
Optionally provide a name for the Karpenter Node IAM Role. If unset, a name will be generated.
null
Optionally provide a path to the Karpenter Node IAM Role.
null
ARN of the policy that is used to set the permissions boundary for the role.
null
karpenter_node_iam_role_tags
map(string)Additional tags to add to the Karpenter Node IAM Role.
{}
Optionally use an IAM name prefix for the Karpenter IAM Role.
false
Optionally provide a name for the Karpenter service account that will be associated with IRSA.
"karpenter"
The ARN of the Karpenter Controller IRSA Role.
The Name of the Karpenter Controller IRSA Role.
The ARN of the Karpenter Node Instance Profile.
The ARN of the Karpenter Node IAM Role.
The name of the Karpenter Node IAM Role.