Rate Limiter System Design Interview

Design a rate limiter that sits on the API gateway path and decides, per request, whether a client may proceed. The hard part is not the definition—it is shared counters across many gateways, a fast check, and a clear fail-open vs fail-closed policy when the store blips.

In production, rate limiters sit in front of APIs (often at the gateway) and decide per request whether to allow or reject based on the client’s usage so far. You need to support many clients, many gateway nodes, and strict latency and availability requirements. The system should support configurable limits per client (e.g. by user ID, IP, or API key), per endpoint, and per tier (e.g. free vs premium). You should be able to explain one counting style, how state is shared across nodes, and what happens when the rate limiter or its store fails.