Fix the design · Incident triage
Help Hari Finish the Rate Limiter
Hari froze mid-interview after drawing per-server in-memory counters for a 100 requests/minute per-user limit that must hold across many API servers.
Hari is interviewing at Nimbus. The interviewer asked him to design a rate limiter. They locked requirements, he drew a rough sketch, then got stuck. Help him answer the questions and update the design.
We need your help. Identify what is incomplete or wrong in Hari’s sketch, then redesign the path from “incoming API request” → “allow or reject” so the 100/min per-user limit holds across many servers, short bursts still work, and clients get a clear retry signal.
Problem
Help Hari finish Nimbus’s rate-limiter design — the team already agreed on requirements, and he has a rough high-level sketch.
Incident summary
- Hari’s sketch counts limits in memory on each API server — with many servers the real limit drifts.
- He only named a fixed one-minute window and did not explain bursts, refill, or which key to count on.
- The deny path is missing: no clear 429 or Retry-After story for clients.
- He has no answer yet for what happens if the counter store is briefly down.
- He never asked how big the counter cache gets, or how to change the rate or key (user id → route/tenant) without a redeploy.
Impacted services
- Limit key (unknown) — Hari never said what identity to count on (user id vs IP).
- In-memory counters (critical) — Each API server counts alone — the real limit multiplies with server count.
- Deny path (degraded) — No 429 or Retry-After drawn for clients who are over the limit.
- Store failure (unknown) — No answer yet for a brief counter-store outage.
Triage questions
- What identity should Hari’s rate limiter count on, and why does that match the locked requirements?
- What is wrong with counting in memory on each API server when many servers sit behind one load balancer?
- How would you redesign Hari’s sketch so the 100/min limit holds across servers and still allows short bursts?
- The shared counter is on every request path. What bottlenecks worry you, and how would you keep limits and the rate-limit key configurable?
- If the shared counter store is briefly down, should Hari allow all traffic or block — and what would you defend in the interview?