High Throughput Architectures on EKS

If you are running an ad technology (AdTech) or game server that receives millions of requests per second (RPS) and requires low latency, standard EKS settings will not be enough. You need a special architecture to overcome network bottlenecks.

1. Using Network Load Balancer (NLB)

HTTP-based Application Load Balancer (ALB) needs time to scale after a certain load (requires "Pre-warming"). Network Load Balancer (NLB) can handle millions of requests instantly. It manages traffic at the L4 (TCP/UDP) level and is much faster.

2. Pod Network Performance

• Prefix Delegation: By enabling this feature in the AWS VPC CNI plugin, you can assign more IP addresses (and thus more pods) to a node. This reduces API load on network calls.
• SR-IOV and ENA: Choose high-performance EC2 instance types (e.g., c5n, m5n). These servers offer network bandwidth up to 100 Gbps with Elastic Network Adapter (ENA).

3. eBPF Instead of Kube-Proxy

Standard kube-proxy (iptables mode) can slow down when there are many services. By using eBPF-based CNI plugins like Cilium, you can route packets much faster at the kernel level.

These techniques turn EKS into a Formula 1 car.