Fix the design · Incident triage

Rate Limiter Gap: The Config That Let One Tenant DDoS the API

On Razorpay’s shared API gateways, one misconfigured enterprise key used about 80% of capacity for 45 minutes, starving other merchants with about 22% too-many-requests errors.

You are on Razorpay’s API platform team. One enterprise API key had no real limit and used about 80% of gateway capacity. Other merchants got too-many-requests errors for 45 minutes.

We need your help. Identify the bottlenecks and failure modes in the current design, then redesign the path from “merchant request” → “allowed or rejected” so one noisy key cannot starve others, counters stay consistent across gateway nodes, and a Redis blip cannot turn all limits off.

Problem

Redesign Razorpay’s existing public API rate-limit path — it already gives each merchant an API key and shares gateway capacity across many tenants.

Incident summary

  • For 45 minutes one API key used about 80% of shared gateway capacity.
  • Other merchants were starved — about 22% of their requests hit too-many-requests errors.
  • Each gateway node cached counters locally, so nodes disagreed on usage and limits drifted.
  • A brief Redis outage turned limits off (fail-open) for about 90 seconds.

Assumptions made by the team

  • A per-key limit is enough to keep tenants fair.
  • Caching counters locally on each gateway node is fine for speed.
  • Turning limits off when Redis struggles keeps the API available — and that is acceptable.
  • One token-bucket setting works for every kind of traffic.

Impacted services

  • API gateway (critical) — Most capacity went to one misconfigured key.
  • Redis counters (degraded) — Went down briefly — limits turned off for about 90 seconds.
  • Other merchants (critical) — About 22% of their requests were rejected as too many requests.
  • Enterprise key owner (degraded) — Kept sending traffic with no effective ceiling.

Triage questions

  1. What is wrong with sharing one rate-limit pool across all tenants without per-tenant caps?
  2. How would you redesign rate limiting so one noisy customer cannot starve everyone else?
  3. When the limiter store blips, should traffic be blocked or allowed through — and why?
  4. How would you keep rate-limit counters consistent across many gateway nodes?

← All design challenges

Loading scenario…