Kubernetes Scheduler Logic and EKS Behaviors

When you create a pod, kube-scheduler decides which Worker Node it will run on. This decision process directly affects Amazon EKS performance and cost.

Scheduling Process

The Scheduler performs a two-stage elimination for each new pod:

1. Filtering: Eliminates nodes where the pod cannot run.
   • Is there enough CPU/RAM?
   • Do the Taints on the Node match the Tolerations of the Pod?
   • Do Node Selector and Affinity rules match?
2. Scoring: Assigns scores to the remaining nodes.
   • Image Locality: If the pod's image is already on this node, score increases (starts faster).
   • Least Requested: The emptiest node usually gets a higher score (Load balancing).

Things to Consider in EKS

• AZ Distribution: If you don't define topologySpreadConstraints, Scheduler might pile pods into the same AZ. This is an HA risk.
• Bin Packing vs Load Balancing: Scheduler tries to distribute load by default (Least Requested). However, if you want to pack pods (Bin Packing) for cost optimization (Karpenter), you might need to change Scheduler profiles.

The reason your pods stay "Pending" is usually because the Scheduler cannot find a suitable node. The kubectl describe pod <pod-name> command clearly tells you why the Scheduler failed (e.g., Insufficient cpu).