Rate Limiter System Design Interview

IntroductionRead out

A rate limiter sits in front of your API and asks, for every request: is this user still under their limit? Yes → continue. No → stop them.

That check usually runs on the API gateway (the front door). You have many front-door servers. If each keeps its own private count, users get far more than you promised. So every server must use one shared count.

Example: a phone retries checkout twenty times in one second. Without a limiter, you might charge them twenty times. With a bad limiter (private counts per server), they can still blow past the real limit.

This page is the interview deep dive in the system design interview guides hub. Brand new to the idea? Skim the fundamentals series first.

Why teams bother with rate limitingRead out

You’ve already felt this as a user. After too many wrong passwords, login locks you out for a while. Email apps stop you from sending hundreds of messages in a minute. Apps that offer a public API often say “you may call us 1,000 times per hour.” Same idea: protect the system, and keep one noisy user from hurting everyone else.

Board path (first read, ~10 min)Read out

Start here if you have never designed a rate limiter. Finish this section, then stop and try a mock. Deep Redis, algorithms, and capacity live in After one mock.

What problem are we solving?

Before each request hits your real service (checkout, login, search), ask:

“Has this user already used up their allowance for this minute (or hour)?”

• Under the limit → let the request through.
• Over the limit → stop them. Send HTTP 429 (“too many requests”) and tell them how long to wait (Retry-After). That is different from 503, which means “our service is broken,” not “slow down.”

That question needs a count of recent calls. Easy on one server. Hard when you have many front doors—each with its own memory.

Why one shared count (not a count per server)

Imagine the product promise is: 100 requests per minute per user.

You have 50 front-door servers (gateways). If each server keeps its own private counter, each one thinks “this user still has room until I see 100.” Across 50 servers, that user can sneak through about 50 × 100 = 5,000 requests per minute—fifty times what you promised.

So every server must look at the same counter for that user—one shared count—not fifty private ones.

What to write on the board (start simple)

Write these four lines. Plain English is enough for a first mock.

1. Who? — Who are we counting? Example: “logged-in user + which API route.” Write one example name on the board, like checkout:user:123.
2. Where is the count? — One shared place every server reads/updates. Not “memory inside each server.”
3. What happens on allow / deny? — Under limit → call the real backend. Over limit → return 429 + “try again in N seconds,” and do not call the expensive backend.
4. What if the counter store is down? — For login: stop requests (safer). For browse/catalog: you may let traffic through so the site stays up—say that choice out loud.

That’s your whole first board. Names of tools (Redis, algorithms) come after this mock.

Clarifying questions (identity, burst, fail mode, out of scope)

Start with identity—every later choice hangs off who is being limited.

You: “Are we limiting by user id, API key, IP, route, or a combination? If someone is behind a shared office NAT, limiting only by IP can punish many people for one noisy neighbor.”

Interviewer: “Start with authenticated user id per route; you can mention IP as a secondary abuse signal for anonymous traffic.”

Checkpoint: Write the counter key as a string first (for example checkout:user:123). Don’t draw boxes until the key is on the board.

Shared office NAT: 200 employees behind one IP means IP-only limits punish innocents—keep IP as a coarse anonymous signal, not the primary logged-in budget.

You: “If a phone retries a few times on a bad network, can they go a little over the steady limit for a short moment—or is the limit strict every minute with no extras?”

Interviewer: “Yes—short bursts for retries are OK.”

So the product allows a small spike. You’ll pick a counting style that fits bursts later—not on this first board.

You: “If the shared store times out, should /login and /catalog behave the same?”

Interviewer: “You choose—and defend it.”

Fail-closed = block when the store is down (usual for login / password reset). Fail-open = let traffic through (sometimes OK for catalog reads). Checkpoint: Name different policies for /login vs /catalog out loud.

You: “I want to keep network DDoS scrubbing, a long-term analytics warehouse, and client-only throttling out of scope so we can focus on server-side request limiting in the gateway. Does that match what you want?”

Interviewer: “Yes—focus on server-side request limiting.”

If the interviewer asks about exact counts, store brand, or a tiny per-machine brake—park it. Answer after your requirements mock; those details live under After one mock.

What you can say out loud (next to the board)

1. Who: “I’d count logged-in user plus route. I’d only use IP for anonymous traffic, because office Wi‑Fi shares one IP.”
2. Bursts: “I’d ask if short bursts are OK when a phone retries—many products want that.”
3. Shared count: “I’d keep one shared count for all servers, and update it in one step so two servers can’t both approve the last request.”
4. Deny + outage: “Over limit → 429 with wait time. If the counter store is down: block login; catalog may stay open.”

Why two gateways need one-step updates

Picture two ushers both selling the last concert seat because they each looked at the remaining count before either updated it.

t0  Gateway A reads remaining → 99
t0  Gateway B reads remaining → 99
t1  Both allow → 100th and 101st both go through

Property: the shared count must update in one indivisible step so those reads cannot interleave. (After one mock you’ll name common tools—increment-with-expiry or a short server-side script. You don’t need tool names for the first mock.)

Common traps (remember these before you practice)

Private counts per server blow the limit.
If each of 50 servers allows 100/min in its own memory, one user can reach about 5,000/min. Keep one shared count.

“Per minute” can spike at the clock edge.
At 12:00:59 someone can use the last of the old minute; at 12:01:00 a new minute starts. In a few seconds they may send about twice the advertised rate. Say that out loud if you use strict clock minutes.

Store-down behavior depends on the route.
Let traffic through keeps the site up but is risky for login. Block protects login but can look like an outage for browsing. Don’t use one rule for every route.

A bare 429 makes clients retry harder.
Tell them how long to wait (Retry-After). 429 = slow down; 503 = something is broken—not the same.

First-pass: what should stick

1. One shared count — Private counts on each server multiply the published limit (50 servers × 100/min ≈ 5,000/min).
2. Update in one step — Don’t let two servers both read “1 left” and both say yes.
3. Stop cleanly — Over limit → 429 + wait time, before the expensive backend. Login: block if the counter store is down. Catalog: you may keep it open—say so.

End of first pass

---

⏸ End of first pass — stop scrolling.

You have enough for the requirements stage. Practice that now. Do not continue into Redis, algorithms, or capacity until you’ve done a short requirements mock.

→ Practice requirements only (~5–10 minutes)

---

Ready for depth? Only then open After one mock.

After one mockRead out

Return to Board path if this is your first read.

Everything below is for after one mock—or when the interviewer pushes on algorithms, Redis, capacity, and production failure modes. Optional fundamentals: rate limiting series (fundamentals, algorithms).

Depth map (read in order):

1. How to approach (full clarifying + pacing)
2. Capacity (why the check must stay simple)
3. High-level architecture (one request path)
4. How will you design the rate limiting library (API + counting style)
5. Data model + Dig (race lab: atomic Check, then push topics)

Skip ahead only if the interviewer is already there.

Where the shared count usually lives

You already know you need one count every server sees. In interviews that count often lives in a fast shared store (commonly Redis). The brand is optional—the property that matters is “one source of truth.”

Updates must happen in one step (atomic). Optional later: a tiny per-machine brake in front of the real shared budget (sometimes called a hierarchical limit—local first, shared second).

If they push on exactness: many teams accept a small over-allow (for example ≤5–10% in a minute) if backends stay protected. Fail-open catalog paths may add that tiny local safety brake—not a replacement for the global budget.

Interviewers also want a named algorithm, honest HTTP behavior, and fail-open vs fail-closed as product choices.

How to approachRead out

Spend the first few minutes turning a vague prompt into a contract. Talk through clarifying questions out loud, then walk one request end to end before you name Redis commands.

How to design a rate limiter in an interview

1. Clarify who is limited (user, API key, IP, route), burst vs rigid window, exactness, and fail-open vs fail-closed per route.
2. Estimate check QPS, distinct keys, memory per key, and a sub-ms latency budget.
3. Pick an algorithm from burst shape: fixed window (cheap, edge spikes), token bucket (product bursts), or sliding window (boundary fairness).
4. Decide where the count lives. Many gateways ⇒ one shared counter per client. Sketch the key string. In interviews this shared store is often Redis.
5. Design one Check path:
   • 5a. Allow → forward; deny → 429 + Retry-After before upstream.
   • 5b. Update the count as one step (not read-then-write).
   • 5c. Tools when asked: increment-and-set-expiry (INCR + EXPIRE), or a small Redis script (Lua).
6. Cover hot keys, layered limits, what you measure, and one production failure mode.

Turn board answers into Redis and algorithm choices

Do not re-run the board-path dialogues. You already locked identity, burst-for-retries, fail mode, and out of scope under Clarifying questions (identity, burst, fail mode, out of scope). This section is only how those answers unlock a Redis key, an algorithm name, an exactness bound, and a local safety brake.

From identity → a store-ready key.
Board answer: authenticated user id per route (IP only as a coarse anonymous signal).
Now write the counter as something Redis can hold—for example rl:tb:checkout:user:123 (rate-limit + token-bucket + route + principal). Put that string on the board before you draw boxes.

From burst → an algorithm you can defend.
Board answer: short bursts for mobile retries are OK.
That is your reason to prefer token bucket as the primary pick. Fixed window alone is a weak primary when the product wants spikes. Keep sliding window in reserve when minute-boundary fairness is the real pain.

You: “At peak, do we need exact counts, or is a small over-allow acceptable if backends stay protected? Exact sliding windows cost more Redis work per check.”

Interviewer: “Protect backends; a small over-allow is OK if you say the bound.”

A sayable bound: roughly ≤5–10% over a one-minute window is often acceptable if backends stay healthy. That bound is new depth—the board path parked exactness on purpose.

From fail mode → a local shadow cap (only if you fail open).
Board answer: fail closed on login / password reset; catalog may fail open.
When the shared store is Redis and catalog fails open, add a local shadow cap: a short-lived in-memory brake on that one gateway. It is not the global budget—only a shock absorber so one machine cannot melt while Redis is dark.

Out of scope stays what you already agreed: no network DDoS scrubbing, no analytics warehouse, no client-only throttling as the control plane. Don’t re-ask it—just keep that list on the board.

Minute pacing (about 40 minutes)

Minutes	Focus
0–5	Clarify key, burst, exactness, fail mode, out of scope
5–12	Capacity + high-level boxes + one request story
12–25	Algorithm pick + Redis keys + atomic check (the dig)
25–35	Failure, hot keys, headers
35–40	One production war story + questions

In the room (fuller opening after the board): “I’d start by agreeing who we limit—usually logged-in user id plus route, with IP only as a coarse signal for anonymous abuse. I’d ask whether the product needs short bursts for retries; if yes, I’d lean token bucket, and I’d keep sliding window in reserve when minute-boundary fairness matters. I’d put the counter in a shared store (often Redis) and update it in one indivisible step so two gateways cannot both approve the same last request. On failure I’d fail closed for login and password reset, and I might fail open for catalog reads with a tiny local safety brake. Every deny returns 429 with Retry-After so clients back off instead of hammering harder.”

If you remember one thing: Tell one request story before you recite command names.

Out of scope (and why you park these)

Write a short out-of-scope list on the board so the interviewer sees you can bound the problem.

Network-layer DDoS mitigation (CDN, WAF, scrubbing). Those systems absorb volumetric floods before traffic reaches your gateway. Different interview, different team.

Client-only rate limiting as the only control. Apps can be modified or ignored. Server-side enforcement is what makes the limit real.

A long-term analytics warehouse for every allow or deny. Useful later; building it here burns the round without proving you can protect the request path.

Perfect globally exact counts across every region with zero latency cost. Prefer regional budgets, sticky routing, or an explicit over-allow window—and say which accuracy you are buying.

Capacity estimationRead out

The check must finish so fast users don’t feel it (often under a millisecond). That means a few simple lookups—not asking every server to vote on every request.

Rough numbers to anchor the whiteboard (tune in the room):

Input	Order of magnitude	Design implication
Aggregate checks/s	10M+ (illustrative scale)	Prefer a handful of simple key lookups—not heavy “search the whole history” work on every request
Distinct keys	10²–10⁸ (clients × policies)	Memory per key × expire idle keys with the window
Hot key QPS	Single client abusive or NAT	One busy client can overload one slice of the store—plan to split that client’s counters or add a local + shared budget
Check budget	Sub-ms p99 on gateway	Most of the time budget is the network hop to the shared store; keep connections ready

What the numbers imply

• ~10 GB of counter state at 100M keys × ~100 bytes—before replicas. Memory is a first-class design input.
• Expire idle keys with the window, or memory grows forever.
• Don’t ask every server to agree on every request—too slow; use one fast shared counter. (If someone says consensus, that’s the slow pattern you’re avoiding.)
• Prefer simple key operations, batching several commands on one connection (pipelining), and when needed a short server-side script so multi-step checks stay one round trip.
• Isolate hot keys early; one abusive principal can starve one store node before average cluster CPU looks bad.

Checkpoint: What’s your check budget, and why not a multi-server vote per request?

If you remember one thing: Sub-ms budget means simple key ops—not asking every server to agree on every request.

High-level architectureRead out

How to approach this on the board

Do not start by listing every box. Draw one request left to right, then add the shared counter, then (only if asked) where humans edit limits.

1. Client → load balancer → API gateway. Say the gateway terminates TLS, routes, and often authenticates.
2. Before expensive upstream work, the gateway asks: “Is this caller still under limit?”
3. To answer that, it needs one shared count (not a private counter per gateway). In interviews that store is usually Redis.
4. Allow → call the real backend. Deny → return 429 with Retry-After and stop—no checkout/login work.
5. Only after that path is clear, add a small side note: humans set rates elsewhere; gateways keep a cached copy of those rules.

Say this out loud while you draw: “Same shared count for every gateway; check finishes before we spend money on the backend.”

Two jobs (keep them separate)

Think of the limiter as two different jobs—not one blob.

Job	In one sentence	How often
1. Check	“May this request go through?”	Millions of times per day
2. Set rules	“What is the limit for free vs premium, and what do we do if Redis is down?”	A few times a day (or less)

Why split them?
If you mix them, you draw one messy box and then cannot answer: “Where does a PM change 100/min?” vs “What happens on every checkout call?” Interviewers push on both. Separating the jobs keeps the hot path tiny (fast check only) and puts slow human changes off to the side.

Job 1 — Check (every request):
Build a key → read the cached rule → one shared-count update → allow (with headers) or 429. This must stay fast.

Job 2 — Set rules (humans / ops):
Someone changes rate, burst, or fail-open vs fail-closed (config service or feature flags). Gateways cache those rules. Changing a limit is not a Redis write on every request.

You can say “check path” and “policy path.” Labels like data plane / control plane are optional—only use them if the interviewer does.

Boxes you draw (who owns what)

Box	Owns	Does not own
API gateway	Front door: TLS, routing, auth, first abuse signals; calls the limiter before upstream	The global counter truth
Rate limiter (library or thin service)	Builds the key, applies cached policy, talks to Redis, formats 429/headers	Long-lived product config UI
Redis (shared store)	Authoritative counters; keys spread across a cluster (many nodes). Prefer the primary for increments	Deciding product rates
Policy / config service	Create/update limits, canary a new rate, emergency global throttle	Per-request allow/deny math

Library vs separate limiter service: Prefer an in-process library on the gateway calling Redis—fewest hops. Use a dedicated limiter tier only when many languages share one fleet and you need one implementation.

Measure or you’re blind: track check latency (p99) and allow vs deny counts. When Redis slows down, those metrics tell you before users only feel “site is weird.”

Figure: Sync path = allow/deny against Redis. Async path = policies refresh into the gateway cache; analytics never block the check.

In the room: Narrate one request end-to-end: key string → one Redis round trip (or Lua) → 200 with headers vs 429. Then: “If Redis times out, login fail-closed; catalog may fail-open with a tiny local brake.”

If you remember one thing: Draw the check before upstream first; Redis holds the shared count; policy edits are a side path cached on the gateway.

Follow-ups the interviewer may ask

After you draw the boxes, they often poke at ownership and failure—not algorithms yet. Short answers:

Interviewer: “Why not put the counter in each gateway’s memory? It’s faster.”

You: “Fifty gateways with a private count of 100 each become a 5,000/min limit. Memory is fine as a tiny local shock absorber; the shared Redis count is still the product promise.”

Interviewer: “Library on the gateway vs a separate rate-limiter microservice?”

You: “I’d start with an in-process library calling Redis—one less hop. I’d only add a dedicated limiter tier if many languages share one fleet and we need a single implementation.”

Interviewer: “Where do product managers change ‘free = 100/min’ without redeploying every gateway?”

You: “That’s the policy path—config service or feature flags. Gateways cache the rules with a short TTL. Changing a rate is not a Redis write on every request.”

Interviewer: “Redis is slow or times out—does the whole site die?”

You: “No—not the whole site. You choose per route. For login and password reset, block when Redis is down (safer—don’t leave the door open for password guessing). For catalog browsing, you may let traffic through so the site still works, plus a tiny local limit on that one machine as a safety net. Also watch how long the check takes—if it gets slow, metrics show it before users only feel ‘the site is weird.’”

Interviewer: “Can analytics or Kafka sit on the allow/deny path?”

You: “No. The allow/deny check has to finish before we call the real backend—and it has to stay fast. If every request waits on Kafka or an analytics write, Redis being healthy won’t matter: the queue slows down, fills up, or fails, and now checkout waits on logging. Decide first (build the key, one atomic Redis check, return 200 or 429). After that decision, you can sample or ship a small event to Kafka asynchronously—logging must not sit in front of the decision.”

How will you design the rate limiting libraryRead out

After the boxes are on the board, interviewers often zoom into the library (or thin service) that every gateway calls: what is the API, which counting style, and how you keep the check one fast step against Redis.

Interviewer may ask

Interviewer: “How would you design the rate limiting library the gateway calls on every request?”

You: “One small API: given who is calling and which route, return allow or deny plus headers. Inside: build a key, load the cached policy, run one atomic check against the shared store, then format 200 headers or 429 + Retry-After. The gateway stays dumb—it just invokes Check before upstream work.”

Interviewer: “What does that API look like?”

You: “Something like Check(principal, route) → { allowed, remaining, limit, reset_at, retry_after_sec }. Optional weight for expensive calls. Fail mode (fail-open vs fail-closed) comes from the policy, not hardcoded in the library.”

Interviewer: “Where does the counting algorithm live—gateway, Redis, or both?”

You: “The library chooses the algorithm from the policy, but the update must run as one step in Redis (INCR+EXPIRE or a short Lua script). If the library does GET then SET in two round trips, two gateways can both approve the last request.”

Interviewer: “Which algorithm do you ship first?”

You: “I’d ask about bursts. If mobile retries need a short spike, token bucket. If they only need a coarse abuse cap, fixed window. If minute-boundary fairness causes support tickets, sliding window (exact or approximate). I pick from burst shape and work per check—not the most impressive-sounding name.”

What the library owns

Responsibility	In the library	Outside the library
Build tenant:route:user (or similar) key	Yes	—
Load rate / burst / fail mode	From cached policy	Policy store / control plane
Atomic allow/deny	Yes (calls Redis)	Redis holds counters
HTTP 429 + Retry-After + limit headers	Formats decision	Gateway returns the response
TLS, routing, auth	No	API gateway

Prefer an in-process library on the gateway. A dedicated limiter service is the same design with an extra network hop—only justify it when many languages need one shared implementation.

Decision matrix (pick the counting style)

Token bucket vs sliding window vs fixed window—pick from burst shape and work per check.

Algorithm	Burst behavior	Work per check (shared store)*	Fairness at window edge	When to pick
Fixed window	Worst—can allow ~2N near boundaries	Lowest (increment + TTL)	Poor	Coarse abuse caps
Sliding window (exact)	Smooth	Highest	Best	Support-ticket magnet on fairness
Sliding approx	Near-smooth	Medium	Good enough	High QPS with admitted error
Token bucket	Smooth up to capacity B	Medium (server-side refill)	Good	API products that promise bursts
Leaky bucket	Smooth outflow; queues or rejects	Medium	Steady rate	When you want constant drain, not burst

*In interviews that shared store is often Redis.

Interview tip: After you pick, say burst shape + work per check in one sentence—that is how the library’s first algorithm earns its place.

The others in one breath (then go deep on token bucket)

Use the matrix above to pick. Details for fixed / sliding / leaky live in Rate limiting algorithms—here is only what you say on the board:

• Fixed window — Cheapest (INCR + TTL). Say the edge risk out loud: near a minute boundary a client can get about 2N in a few seconds.
• Sliding window — Fairest at boundaries; exact costs more Redis work. At huge QPS, admit an approximation bound (often ≤5–10%).
• Leaky bucket — Steady drain into a fragile downstream (flat processed rate), not product “bursts.”

Figure: Minute boundaries can double a “100/min” promise in a two-second edge burst—why fixed window alone is a weak primary pick for API products.

Token bucket

Ship this first when the product wants short bursts.

Need: Refill tokens and spend one without another gateway sneaking in between.
Property: One indivisible check-and-update.
Tool: When they ask how, say Redis Lua (or another atomic server-side step)—name the property, then the tool.

Keep a small bucket of tokens. Tokens refill at a steady rate r; the bucket holds at most B (the burst). Each allowed request spends one token (or a weight). Empty bucket → deny (429).

Store remaining tokens and last update time. On each check: refill as if time passed, then try to spend as one server-side step.

  tokens ████████░░  capacity B
  refill →→→ at rate r
  each allow consumes 1 (or weight)
  tokens == 0 → deny (429)

Ship token bucket as the primary when the product wants short bursts; keep fixed or sliding as a named fallback. Deep dive: Rate limiting algorithms.

If you remember one thing: The library is a small Check API plus one atomic Redis update; pick the algorithm for burst shape and work per check—not because the name sounds impressive.

Data model (Redis keys and policies)Read out

Without a key schema, “use Redis” is not a design. Think of each Redis key as a named string that points at a small value (a count, or a few fields)—like a row id for one client’s budget.

One checkout request first: User 42 hits /checkout. Key string: rl:tb:checkout:user:42. Value: remaining tokens + last update time. The who part (user:42) is the principal; the checkout rule set is the policy. The table below is just other shapes of the same idea.

Counter keys (per-request store)

Key pattern	Example	Fields / value	TTL
Fixed window	rl:fw:{policy}:{principal}:{yyyyMMddHHmm}	integer count	~2× window
Token bucket	rl:tb:{policy}:{principal}	hash: tokens, updated_at_ms	idle eviction (e.g. hours)
Sliding approx	rl:sw:{policy}:{principal}:{slot}	counts per sub-window	window + 1 slot
Hot-key shard	rl:tb:{policy}:{principal}:{shard}	same as parent algo	same

Principal = who you already put in the key (user:123, key:abc, or ip:1.2.3.4). Policy = which rule set (route and tier) so /login and /search do not share one counter by accident. TTL is often a bit longer than the window (for example ~2×) so a slow clock edge does not delete the key mid-read.

Policy records (config / control plane)

Store separately from counters (config DB or feature-flag payload), then cache on each gateway:

policy_id: "checkout_user_v3"
version: 17
algorithm: token_bucket
rate_per_sec: 10
burst: 50
scope: "route:/checkout + principal:user"
fail_mode: fail_closed   # or fail_open

Gateways should include version in metrics so mixed deploys are visible. Emergency overrides (whitelist / blocklist) are short-TTL keys checked first.

Example — hierarchical budget (local + Redis)

Published limit: 1,000 req/min for user:42.

• Local gateway brake: ~50/sec on this machine only (shock absorber; can slightly over-allow across many gateways for a few seconds).
• Shared global key: rl:tb:checkout:user:42 enforces the real 1,000/min.
• If one user still overloads one machine in the store: Their counter usually lives on one node. Split that user’s budget across several keys (for example …:user:42:0 … …:user:42:15) so check traffic isn’t glued to one machine. (In Redis Cluster terms, that node ownership is a hash slot—optional label.) Splitting into 16 sub-keys must still sum to the same 1,000/min product limit—not 16 × 1,000.

Local alone is not the product limit; Redis stays authoritative. Prefer checking the cheapest reject first, then the global key—ideally in one network round trip (one Redis script), not N separate waits.

Architectural dig: one Check() callRead out

Board path and High-level architecture already drew the happy path. How will you design the rate limiting library already owns the Check API. This dig answers one question: when two gateways hit Redis at once, what breaks—and how do you say the fix?

How to read this dig: Need first, then property, then tool. You do not invent Redis Lua from scratch in the room—name “one indivisible check-and-update.” When they ask how, say Redis Lua or INCR+EXPIRE.

Bad design: GET then SET

t0  Gateway A: GET count → 99
t0  Gateway B: GET count → 99
t1  A: SET 100   B: SET 100
    both allowed the 100th and 101st → over-allow

Both machines read “99 left,” both thought they could take the last slot, both allowed. The shared count must update in a way that cannot interleave.

Great design: one round-trip atomic update

Need: Two gateways cannot both take the last slot.
Property: One indivisible check-and-update.
Tool: INCR + EXPIRE for fixed windows, or a Lua script for token-bucket refill + consume—one Redis round trip.

-- One Redis Lua round trip (conceptual — not runnable Redis)
1) Read tokens + updated_at from HASH
2) Refill: tokens = min(B, tokens + r * elapsed_seconds)
3) If tokens < cost → DENY + retry_after
4) Else tokens -= cost; write HASH → ALLOW + remaining

In the room: “Atomicity is the design—never GET then SET for a global cap.” If you said GET then SET, correct yourself to Lua or INCR before they push.

Redis is popular in interviews because multi-step updates can run as one server-side script. Without that, you fight the same race with compare-and-set loops—harder under load.

Push topics (when they dig past the race)

Weighted cost: A 100 MB upload might cost 10 tokens while a metadata GET costs 1—so tiny calls cannot burn the whole minute while heavy work starves (or the reverse). State the weight rule when they mention uploads or GraphQL cost. (Refill basics live under Token bucket.)

Idempotency and charging: Mobile clients retry. If every retry burns a token, a flaky network throttles a user who never finished the write. Ask: “Do duplicate POSTs with the same Idempotency-Key consume quota once or every time?” Strong answer: charge once when the backend dedupes that key; still charge distinct logical operations. Do not invent silent free retries without stating the product rule.

Redis wipe (one line): Counters reset → brief over-allow. Accept it, optionally tighten limits for a few minutes, alert on a sudden drop in 429s with a traffic spike—see Failure handling. Hot keys and Lua CPU belong in Key challenges; pool wait and check p99 belong in Observability and Bottlenecks and tradeoffs.

If you remember one thing: Dig is the race lab—GET then SET is the red flag; one atomic Redis step is the fix.

Key challengesRead out

• Correctness vs cost: Exact sliding windows at 10M checks/s per cluster may require approximation; admit error bounds (for example ≤5–10% over a minute).
• Hot keys: One busy client can send so many checks that one Redis node (the one that owns that client’s key) hits 100% CPU while the rest of the cluster looks fine. Mitigation is part of the design (split keys, isolate tenants)—not an afterthought.
• Layered limits: Per-user + per-IP + global must compose without N sequential waits—one Redis script or batched keys.
• Clocks: Enforce windows with server time inside the shared store, never the client’s phone clock. Across regions, small skew can shift window edges; prefer token-bucket state (tokens + updated_at) that refills from elapsed server time. If the interviewer says “NTP skew,” answer: “Client clocks don’t enforce; we tolerate small server skew or centralize the counter.”
• Multi-tenant fairness: A noisy neighbor on shared Redis can raise everyone’s latency—quotas, isolation by tenant prefix, or dedicated clusters for large customers.
• Multi-region consistency: If each region has its own Redis, a user who hops regions can get roughly R × the limit for a short window unless you keep them on one region’s counters (sticky routing), size regional budgets for that drift, or use a central authority for strict global caps. Sayable example: 3 regions, each with its own Redis, limit 100/min, no sticky routing → a hopping client can briefly approach ~300/min. Say the over-allow window you accept instead of claiming perfect global exactness.

If you remember one thing: Hot keys and layered limits are design requirements—not production surprises.

Scaling the systemRead out

• Shard Redis by hash(key); high-cardinality principals usually spread naturally. Hot OAuth clients still need sub-keys or isolation.
• Horizontal gateways scale statelessly; Redis scales out until ops cost hurts—then a dedicated limiter tier with batching.
• Regional limits: stale cross-region counts are acceptable for many products; global strict caps need central authority or sticky routing—name latency vs accuracy.
• Read replicas for limiter increments are risky—prefer primary for updates unless you design probabilistic local caches on purpose.

If you remember one thing: Gateways scale statelessly; Redis scales until you need sharding and abuse isolation.

Failure handlingRead out

Scenario	Bad outcome	Mitigation
Redis timeout	Unbounded traffic (fail open) or outage (fail closed)	Stop calling Redis for a short time after repeated failures (circuit breaker); only cache a short “allow/deny” where safe; different policy per route
Redis failover / wipe	Counter reset → temporary over-allow	Accept burst; temporarily lower limits; alert on anomaly
Gateway deploy	Mixed policy versions	Version keys; canary; warm caches before full traffic
Thundering herd after 429	Clients ignore Retry-After	Add jitter (random small delay) in Retry-After; document exponential backoff
Policy cache stampede	Config service floods after every gateway’s cache expires at once	Don’t expire every cache at the same second; refresh in the background before hard expiry; if many requests miss at once, only one fetch should hit the config service (singleflight)

Degraded UX: Users see more 429s or slower responses. An outage is when gateways error without a policy—avoid that for auth paths.

If you remember one thing: Fail-open vs fail-closed is a product/security choice—never “we’ll decide later.”

API designRead out

Rate limiting is usually not a standalone public REST product in the interview—it is behavior on existing APIs. Still, spell out how clients observe limits.

Gateway-injected headers (common pattern):

Header	Role
X-RateLimit-Limit	Max requests per window for this policy
X-RateLimit-Remaining	Decrements on success; can be approximate
X-RateLimit-Reset	Unix time when window resets
Retry-After	Seconds (or HTTP-date) when returning 429

Some stacks are moving toward standardized RateLimit / RateLimit-Policy headers; in the room, naming the older X-RateLimit-* pattern is still widely understood—say you will match whatever the company’s public API already documents.

429 response body: Machine-readable code, human message, optional retry_after_ms—helps mobile clients.

Internal admin API (sketch):

GET    /v1/admin/policies/{id}
PUT    /v1/admin/policies/{id}      # burst, rps, scope
POST   /v1/admin/overrides          # temporary whitelist / blocklist

Request flow (hottest read):

GET /v1/resource
  → Gateway: auth → limiter.Check(key) → Redis Lua
       → 200 + X-RateLimit-* → upstream
       → 429 + Retry-After (stop)

Errors: 401 before limiter if unauthenticated; 429 for throttle; 503 if upstream overloaded—distinct from throttle so clients do not backoff incorrectly.

If you remember one thing: Retry-After on 429 is part of the API contract—not optional polish.

Checkpoint: What does the client do with Retry-After—sleep, then retry with jitter, or hammer immediately?

Observability (what you measure)Read out

Do not watch only “count of 429.” That misses the failure mode where everything is slow and almost nobody is throttled.

Signal	Why it matters
limiter.check p99 / p999	Critical-path tax on every request
Redis error rate + pool wait	Distinguishes “Redis down” from “pool starved”
Top keys by QPS / Lua CPU	Finds hot keys before users blame “the app”
429 rate by route and policy version	Catches bad deploys and mixed versions
Allow/deny sampling (async)	Debug without blocking the hot path

Alert ideas: limiter.check p99 above budget; single key above shard CPU threshold; sudden drop in 429s during a traffic spike (possible fail-open or counter reset).

Security and abuseRead out

Authenticate before you apply user-scoped limits so anonymous callers cannot burn another user’s budget. Prefer API key or user id over raw IP when users share NATs; still keep a coarse IP or edge limit for unauthenticated flood. Stolen keys need fast revoke in the control plane. Do not treat the app limiter as a replacement for WAF or DDoS scrubbing—those sit further out and buy different protection. On login and password-reset paths, fail closed when the store is unavailable so a Redis blip does not become an open credential-stuffing window.

Cost awarenessRead out

Every allow that should have been a deny wastes upstream CPU, database, and support. Every deny that should have been an allow wastes conversion and trust. Redis memory and Lua CPU are real dollars at 10M checks/s—approximation and hierarchical local caps exist to buy protection without paying exactness everywhere. Returning 429 without calling upstream is the first cost win; isolating hot tenants is how you stop one customer from buying pain for everyone else on a shared cluster.

Bottlenecks and tradeoffsRead out

Exactness vs throughput

The tension — Sliding windows feel fair; exact structures cost more Redis work per check.

What breaks — p99 for Check blows the sub-ms budget at 10M checks/s.

What teams do — Approximate sliding windows; token bucket with Lua refill; admit small error bounds.

Say in the interview — Name fairness vs Redis CPU—not “we use sliding window” alone.

Central store vs edge pre-check

The tension — Edge-only is fast; global caps need a remote round trip.

What breaks — Pure edge under-enforces; pure central adds RTT.

What teams do — Hierarchical: local hard cap + Redis global refine in one Lua pipeline.

Say in the interview — Draw two layers when they ask about 50 gateways.

Availability vs abuse on failure

The tension — Fail-open keeps revenue flowing; fail-closed stops credential stuffing when Redis blips.

What breaks — Wrong default on login during Redis outage = security incident.

What teams do — Route-level policy; circuit breaker with short cached decision only where safe.

Say in the interview — “Catalog read fail-open; login fail-closed”—and mean it.

If you remember one thing: The limiter can become the bottleneck that protects nothing if Redis or the gateway queue is ignored.

Interview tipsRead out

Shorter traps for first readers live in the Board path. When the interviewer pushes after your design, defend these three:

Each gateway keeps its own count — your limit multiplies by the fleet

In the room

You might say: “Each API server keeps its own count in memory for every client. That is fast and simple.”

Interview Push: “You have 50 gateways and a published limit of 100 requests per minute per user—how many requests can that user actually get through?”

Land here: Do the math out loud. Each gateway allows up to 100 before its counter fills. Spread across 50 machines, that is roughly 50 × 100 = 5,000 requests per minute—fifty times the product promise. Local memory is not shared; Gateway A never sees what Gateway B already allowed. Put the authoritative counter in Redis (or another shared store) and update it atomically. A tight local hard cap is fine as a shock absorber only if a global Redis check still enforces the real budget.

Redis is down—do you let everyone through?

In the room

You might say: “If Redis fails, we fail open and let traffic through so the site stays up.”

Interview Push: “Is that OK for password reset and login?”

Land here: Fail-open keeps availability but invites abuse—credential stuffing loves an open door. Fail-closed protects auth paths but can look like an outage for catalog reads. Strong answers pick different policies per route: fail-closed on login and password reset; maybe fail-open on read-heavy catalog with a tight local shadow cap. Never treat one global default as “neutral.”

A bare 429 teaches clients to hammer harder

In the room

You might say: “We return HTTP 429 when the user is over limit.”

Interview Push: “What should the client do next—and what happens if you give no guidance?”

Land here: Return Retry-After (seconds or an HTTP-date) plus remaining/limit/reset headers when you can. Without backoff hints, well-meaning clients and broken retry loops retry immediately, which turns a throttle into a thundering herd. Distinct status codes matter too: 429 means “slow down”; 503 means “origin unhealthy”—clients should not treat them the same.

Red flags to avoid without correcting yourself: “GET then SET,” “each node counts locally for a global cap,” “429 with no Retry-After,” “one fail-open policy for every route.”

After each push, close with one real part of the design you would build—a shared Redis key, a Lua check, a route-level fail mode—not “we will scale it later.”

What should stickRead out

First-pass three bullets live in the Board path. After the depth sections, you should be able to explain:

1. Shared state — Global limits need one authoritative counter per key (or an honest hierarchy). Local counters multiply the limit by the fleet size.
2. Algorithm choice — Fixed window is cheap but spiky; token bucket for product bursts; sliding for boundary fairness; leaky bucket for steady drain. Details: Rate limiting algorithms.
3. Atomicity — INCR+EXPIRE or Lua for read-modify-write in one RTT; never GET then SET.
4. HTTP contract — 429 + Retry-After + limit headers; distinct from 503. Deny before upstream.
5. Failure is policy — Fail-open vs fail-closed per route; hot keys need sharding; watch limiter.check p99 and top keys, not only 429 count.

Tell it in the room: “Every request: the gateway builds tier:route:principal, runs one atomic Redis check, then either returns 200 with limit headers or 429 with Retry-After without calling upstream. I’d use token bucket when the product needs bursts. Login fails closed if Redis times out; catalog may fail open with a tight local shadow cap. One hot client id means I shard that counter key so one Redis slot cannot take down the cluster.”

Quick check before a mock: (1) 50 gateways × 100/min local = ? (2) Why is GET-then-SET wrong? (3) Login fail mode when Redis is down? → (1) ~5,000/min (2) two gateways both allow the last request (3) fail-closed.

Practice next: Open Design a Rate Limiter on Practice System Design and run a timed attempt using clarifying questions and one request story—without peeking until you have a draft on the board.

Related TopicsRead out

• System design interview guides - Hub of whiteboard walkthroughs
• Rate Limiting and Throttling - Fundamentals behind gateway limits
• Rate Limiting Algorithms - Fixed window, sliding window, token bucket deep dive
• Load Balancing - Spreading traffic before (and around) the limiter
• Circuit Breakers - Failing fast when Redis or upstreams degrade
• Vaccine Booking System Design - Admission control and flash traffic patterns
• Distributed Cache System Design - Shared in-memory state and hot keys
• Notification Service System Design - Async work that must not block the request path