Skip to main content
Amazon EKS 0.65.5Last updated in version 0.65.3

EKS K8s Karpenter Module

View SourceRelease Notes

This Module can be used to deploy Karpenter as an alternative to the Cluster Autoscaler for autoscaling capabilities of an EKS cluster. This module will create all of the necessary resources for a functional installation of Karpenter as well as the installation of Karpenter. This module does not create Karpenter Provisioners or Node Templates, only the installation of the Karpenter Controller. See the Karpenter Example for an example of how to deploy the additional CRDs (Provisioners, Node Templates, etc) to the EKS cluster.

Note: For EKS cluster autoscaling capabilities, either Karpenter OR the cluster-autoscaler should be used; not both. To migrate to using karpenter instead of the cluster-autoscaler see Migrating to Karpenter from the Cluster Autoscaler

To leverage the full power and potential of Karpenter, one must understand the Karpenter Core Concepts. Deploying this module without additional configuration (ie deploying Karpenter CRDs) will not enable EKS cluster autoscaling. As use-cases are presented, we will do our best effort to continue to add meaningful examples to the examples folder to help ease the complexities of configuring Karpenter. At minimum, one should configure and deploy a default Provisioner and Node Template for just in time node provisioning via Karpenter.

Resources Created

This module will create the following core resources, some of which are optional which are noted in the input variables:

AWS ResourceDescription
Karpenter Node IAM RoleIAM Role used by Karpenter Nodes provisioned by the Karpenter Controller
Karpenter Node Instance ProfileIAM Instance Profile attached to EC2 instances launched by the Karpenter Controller
Karpenter Controller IRSAIAM Role for Service Account (IRSA) to be used by the Karpenter Controller
Karpenter Helm ReleaseKarpenter deployment to EKS via Helm
Karpenter SQS QueueSQS Queue used to listen to EC2 events

Additional supporting resources do exist in addition to the table above, such as IAM Policy Documents, Attachments and CloudWatch rules.


What is Karpenter?

From the Karpenter official docs:

Karpenter is an open-source node provisioning project built for Kubernetes. Adding Karpenter to a Kubernetes cluster can dramatically improve the efficiency and cost of running workloads on that cluster.

Karpenter works by:

  • Watching for pods that the Kubernetes scheduler has marked as unschedulable
  • Evaluating scheduling constraints (resource requests, nodeselectors, affinities, tolerations, and topology spread constraints) requested by the pods
  • Provisioning nodes that meet the requirements of the pods
  • Removing the nodes when the nodes are no longer needed

For additional details and in-depth information on Karpenter, please see the Karpenter Docs Site.


How to enable de Deprovisioning based on EC2 events?

This is used to inform Karpenter of EC2 events that affect the cluster capacity ( Spot Interruption Warnings, Scheduled Change Health Events (Maintenance Events) , Instance Terminating Events, Instance Stopping Events).

This is particularly useful to users that rely on Spot Instances that can be terminated at will.

For more information read the Karpenter Intrerruption section

  • From variables.tf enable create_karpenter_deprovisioning_queuet

Sample Usage

main.tf

# ------------------------------------------------------------------------------------------------------
# DEPLOY GRUNTWORK'S EKS-K8S-KARPENTER MODULE
# ------------------------------------------------------------------------------------------------------

module "eks_k_8_s_karpenter" {

source = "git::git@github.com:gruntwork-io/terraform-aws-eks.git//modules/eks-k8s-karpenter?ref=v0.65.5"

# ----------------------------------------------------------------------------------------------------
# REQUIRED VARIABLES
# ----------------------------------------------------------------------------------------------------

# The AWS Region to create resources in.
aws_region = <string>

# URL endpoint of the Kubernetes control plane provided by EKS.
eks_cluster_endpoint = <string>

# The name of the EKS Cluster that Karpenter will be associated with.
eks_cluster_name = <string>

# The ARN of the EKS OIDC provider. This is required if creating IRSA for the
# Karpenter Controller.
eks_openid_connect_provider_arn = <string>

# The URL of the EKS OIDC provider. This is required if creating IRSA for the
# Karpenter Controller.
eks_openid_connect_provider_url = <string>

# ----------------------------------------------------------------------------------------------------
# OPTIONAL VARIABLES
# ----------------------------------------------------------------------------------------------------

# Optionally create an IAM Role for Service Account (IRSA) for the Karpenter
# Controller.
create_karpenter_controller_irsa = true

# Conditional flag to optionally create the Karpenter Deprovisioning Queue.
create_karpenter_deprovisioning_queue = true

# Conditional flag to create the Karpenter Node IAM Role. If this is set to
# false, then an existing IAM Role must be provided with the
# `karpenter_node_iam_role_arn` variable
create_karpenter_node_iam_role = true

# Conditional flag to optionally create resources in this module.
create_resources = true

# Additional Helm chart values to pass to the Karpenter Helm chart. See the
# official Karpenter Helm chart values file and documentation for available
# configuration options.
karpenter_chart_additional_values = {}

# The Helm chart name for the Karpenter chart.
karpenter_chart_name = "karpenter"

# The k8s namespace that the Karpenter Helm chart will be deployed to.
karpenter_chart_namespace = "karpenter"

# The Helm release name for the Karpenter chart.
karpenter_chart_release_name = "karpenter"

# The Helm repository to obtain the Karpenter chart from.
karpenter_chart_repository = "oci://public.ecr.aws/karpenter"

# The version of the Karpenter Helm chart.
karpenter_chart_version = "v0.24.0"

# Provide an existing IAM Role ARN to be used with the Karpenter Controller
# Service Account. This is required if `create_karpenter_controller_irsa` is
# set to false.
karpenter_controller_existing_role_arn = true

# Additional tags to add to the Karpenter Deprovisioning Queue.
karpenter_deprovisioning_queue_tags = {}

# A tag that is used by Karpenter to discover resources.
karpenter_discovery_tag = "karpenter.sh/discovery"

# ARN of the policy that is used to set the permissions boundary for the role.
karpenter_irsa_permissions_boundary = null

# Use an existing IAM Role to be used for the Karpenter Node Instance Profile.
# This is required if `create_karpenter_node_iam_role` is set to false. This
# should be the ARN of the IAM Role.
karpenter_node_existing_iam_role_arn = null

# Use an existing IAM Role to be used for the Karpenter Node Instance Profile.
# This is required if `create_karpenter_node_iam_role` is set to false. This
# should be the Name of the IAM Role.
karpenter_node_existing_iam_role_name = null

# A description of the Karpenter Node IAM Role.
karpenter_node_iam_role_description = "IAM Role attached to nodes launched by Karpenter."

# Maximum session duration (in seconds) that you want to set for the Karpenter
# Node role. Value can be between 3600 and 43200.
karpenter_node_iam_role_max_session_duration = 3600

# Optionally provide a name for the Karpenter Node IAM Role. If unset, a name
# will be generated.
karpenter_node_iam_role_name = null

# Optionally provide a path to the Karpenter Node IAM Role.
karpenter_node_iam_role_path = null

# ARN of the policy that is used to set the permissions boundary for the role.
karpenter_node_iam_role_permissions_boundary = null

# Additional tags to add to the Karpenter Node IAM Role.
karpenter_node_iam_role_tags = {}

# Optionally use an IAM name prefix for the Karpenter IAM Role.
karpenter_node_iam_role_use_name_prefix = false

# Optionally provide a name for the Karpenter service account that will be
# associated with IRSA.
karpenter_service_account_name = "karpenter"

}


Reference

Required

aws_regionstringrequired

The AWS Region to create resources in.

eks_cluster_endpointstringrequired

URL endpoint of the Kubernetes control plane provided by EKS.

eks_cluster_namestringrequired

The name of the EKS Cluster that Karpenter will be associated with.

The ARN of the EKS OIDC provider. This is required if creating IRSA for the Karpenter Controller.

The URL of the EKS OIDC provider. This is required if creating IRSA for the Karpenter Controller.

Optional

Optionally create an IAM Role for Service Account (IRSA) for the Karpenter Controller.

true

Conditional flag to optionally create the Karpenter Deprovisioning Queue.

true

Conditional flag to create the Karpenter Node IAM Role. If this is set to false, then an existing IAM Role must be provided with the karpenter_node_iam_role_arn variable

true
create_resourcesbooloptional

Conditional flag to optionally create resources in this module.

true

Additional Helm chart values to pass to the Karpenter Helm chart. See the official Karpenter Helm chart values file and documentation for available configuration options.

Any types represent complex values of variable type. For details, please consult `variables.tf` in the source repo.
{}
karpenter_chart_namestringoptional

The Helm chart name for the Karpenter chart.

"karpenter"

The k8s namespace that the Karpenter Helm chart will be deployed to.

"karpenter"

The Helm release name for the Karpenter chart.

"karpenter"

The Helm repository to obtain the Karpenter chart from.

"oci://public.ecr.aws/karpenter"

The version of the Karpenter Helm chart.

"v0.24.0"

Provide an existing IAM Role ARN to be used with the Karpenter Controller Service Account. This is required if create_karpenter_controller_irsa is set to false.

true

Additional tags to add to the Karpenter Deprovisioning Queue.

{}

A tag that is used by Karpenter to discover resources.

"karpenter.sh/discovery"

ARN of the policy that is used to set the permissions boundary for the role.

null

Use an existing IAM Role to be used for the Karpenter Node Instance Profile. This is required if create_karpenter_node_iam_role is set to false. This should be the ARN of the IAM Role.

null

Use an existing IAM Role to be used for the Karpenter Node Instance Profile. This is required if create_karpenter_node_iam_role is set to false. This should be the Name of the IAM Role.

null

A description of the Karpenter Node IAM Role.

"IAM Role attached to nodes launched by Karpenter."

Maximum session duration (in seconds) that you want to set for the Karpenter Node role. Value can be between 3600 and 43200.

3600

Optionally provide a name for the Karpenter Node IAM Role. If unset, a name will be generated.

null

Optionally provide a path to the Karpenter Node IAM Role.

null

ARN of the policy that is used to set the permissions boundary for the role.

null
karpenter_node_iam_role_tagsmap(string)optional

Additional tags to add to the Karpenter Node IAM Role.

{}

Optionally use an IAM name prefix for the Karpenter IAM Role.

false

Optionally provide a name for the Karpenter service account that will be associated with IRSA.

"karpenter"