Load Balancer System Design (Algorithms, Health Checks & Scale)

Design a load balancer that distributes incoming requests across multiple backend servers to ensure high availability and optimal performance.

Constraints

Functional

Distribute requests across servers, health checks and remove unhealthy servers, optional session affinity, SSL termination, rate limiting per client

Non-functional

Millions of requests/s, minimal overhead (< 1ms), 99.99% uptime, scale with traffic

Scale

10M requests/s, 1000 backend servers, ~1 KB request, ~10 KB response

Stages ahead

1Requirement Analysis
2API Design
3High-Level Design
4HLD Extensions
5Trade-offs