Fix the design · Incident triage

Hot Partition: The Feed That Melted When a Celebrity Posted

On Instagram, a celebrity post to about 40 million followers overloaded one timeline shard at 100% CPU, slowing some feeds to about 45 seconds and leaving about 2.1 million fan-out writes waiting.

You are on Instagram’s feed team. A celebrity with about 40 million followers posts. One database shard hits 100% CPU and some feeds take about 45 seconds to load.

We need your help. Identify the bottlenecks and failure modes in the current design, then redesign the path from “celebrity posts” → “followers see the feed” so one mega-account cannot melt a single shard, fan-out can fall back to pull for huge audiences, and other timelines stay fast.

Problem

Redesign Instagram’s existing home-feed write path — it already shards timelines across many servers and pushes a copy of each post into every follower’s timeline.

Incident summary

  • A celebrity with about 40 million followers posted during peak hours.
  • One timeline shard hit 100% CPU while others stayed under 30%.
  • Feed load times reached about 45 seconds for affected users.
  • Fan-out workers fell behind with about 2.1 million writes waiting.

Assumptions made by the team

  • Pushing to every follower on write is simpler than fetching on read.
  • Sharding by user id spreads load evenly.
  • Celebrity-scale accounts are rare — we do not need a special path.
  • The cache will protect the read path on its own.

Impacted services

  • Timeline shard (hot) (critical) — Hit 100% CPU while sibling shards stayed mostly idle.
  • Fan-out workers (critical) — Fell behind with about 2.1 million writes waiting.
  • Home feed reads (critical) — Some users waited about 45 seconds for their feed.
  • Customers (critical) — Followers of the celebrity saw a broken feed experience.

Triage questions

  1. Why did writing one post into every follower's timeline overload a single shard?
  2. How would you redesign fan-out so a celebrity post does not melt one partition?
  3. Would you push updates to every follower, pull on read, or mix both — and why?
  4. Should celebrity traffic use a separate pipeline from normal posts?

← All design challenges

Loading scenario…