API Gateway System Design (Routing, Auth & Scale)

Design an API Gateway that acts as a single entry point for all client requests to backend microservices. The gateway should handle routing, authentication, rate limiting, caching, and request/response transformation.

Constraints

Functional

Route to backend services, auth (API keys, JWT, OAuth), rate limiting, response caching, request/response transformation, load balancing, logging and monitoring, circuit breaking

Non-functional

< 5ms overhead, millions of requests/s, 99.99% uptime, horizontal scale, secure backend communication

Scale

10M requests/s, 100 backend services, 1M clients; 5 KB request, 20 KB response; 40% cache hit; 50 GB/s in, 200 GB/s out

Stages ahead

1Requirement Analysis

2API Design

3High-Level Design

4HLD Extensions

5Trade-offs