Load Balancer System Design

Design a load balancer that distributes incoming traffic across a pool of backend servers, detects unhealthy nodes, and keeps added latency minimal. In production this sits on every critical path—misconfigured health checks or naive round-robin on long connections cause tail latency spikes that look like application bugs.

You should support request distribution, health-based failover, optional session affinity, SSL termination, and per-client rate limiting. Be ready to compare routing algorithms, explain active vs passive health checks, and describe how you drain connections during deploys. Scaling the LB tier itself and L4 vs L7 tradeoffs are standard follow-ups.