How do you do dynamic scaling based on CPU on EKS?
The EKS cluster autoscaler does scaling based on Pods, but I would like to scale the cluster based on CPU. Right now, I am using the ASG modules directly to manage my worker nodes and implemented dynamic scaling at the ASG level, but it is contending with the cluster autoscaler. Any ideas on how to make the two work together? --- <ins datetime="2022-05-10T14:10:23Z"> <p><a href="https://support.gruntwork.io/hc/requests/108575">Tracked in ticket #108575</a></p> </ins>
In general, it is a bad idea to have two different autoscaling policies for the ASGs, especially if you have multiple systems managing the scaling properties. These autoscaling systems are not designed to work with other autoscalers, and thus you will observe thrashing as the two systems fight to apply their view of the world. For example, the cluster autoscaler only cares about Pod capacity when scaling, so it doesn't care about the CPU load and thus will decide the "right" size of the ASG based on the Pods. The CPU autoscaler on the other hand has no visibility in the Pod capacity, so will decide the "right" size based on the CPUs. If these numbers don't align, then the two systems will have different desired sizes, and will continuously try to modify the state of the world to match what they want, leading to thrashing. --- The typical way dynamic scaling by CPU is handled in the Kubernetes world is to apply the CPU based autoscaling at the Pod level, and then have the Pod autoscaling influence the cluster autoscaler. That is, you can implement [Horizontal Pod Autoscalers](https://kubernetes.io/docs/tasks/run-application/horizontal-pod-autoscale/) on your kubernetes services that will cause EKS to launch more Pods in response to CPU overload. This will in turn lead to resource contention that will prevent the Pods from launching, leading to the cluster autoscaler launching more nodes. Alternatively, you can disable the cluster autoscaler and focus entirely on CPU based scaling. This prevents you from auto scaling the cluster due to other resource contention (e.g., IP addresses), but if you are running out CPU before IPs, then should work well for your environment.