Fix the design · Incident triage

Gateway Overload: The API Gateway That Dropped Half of Traffic

During Swiggy’s dinner rush, a slow inventory service blocked shared gateway threads, returning bad-gateway errors on about 48% of requests for about 22 minutes while health checks still looked fine.

You are on Swiggy’s API gateway team during dinner rush. A slow inventory service blocks shared gateway threads. About half of API requests fail for about 22 minutes while health checks still look fine.

We need your help. Identify the bottlenecks and failure modes in the current design, then redesign the path from “client request” → “backend response” so one slow service cannot exhaust the shared gateway, healthy routes stay isolated, and retries cannot double the load.

Problem

Redesign Swiggy’s existing API gateway — it already routes HTTPS traffic from client apps to backend services such as orders and inventory.

Incident summary

  • The inventory service slowed down — responses took up to about 12 seconds at worst.
  • The gateway returned bad-gateway errors for about 48% of requests over about 22 minutes.
  • TCP health checks still passed because the service accepted connections.
  • Thread pools hit 100% with no queue cap.
  • Client apps retried failed requests and roughly doubled load during the outage.

Assumptions made by the team

  • If the health check passes, the service is fine.
  • A shared thread pool keeps operations simple.
  • Client apps retry carefully.
  • A bad-gateway error always means the downstream service is fully down.

Impacted services

  • API gateway (critical) — Shared thread pools filled up waiting on inventory.
  • Inventory service (critical) — Slow responses up to about 12 seconds blocked callers.
  • Orders service (degraded) — Healthy routes still failed because they shared the same pool.
  • Customers (critical) — About half of API requests failed for about 22 minutes.

Triage questions

  1. Why did one slow backend service take down half of all API traffic?
  2. How would you redesign the gateway so a slow dependency cannot exhaust shared thread pools?
  3. How do client and gateway retries make this kind of outage worse?
  4. Why did health checks stay green while users saw failures — and what would you monitor instead?

← All design challenges

Loading scenario…