Instagram Feed System Design

Introduction

You post a Reel on spotty LTE. The app says "posted" in two seconds—but playback stutters because only a poster frame exists while the full bitrate ladder is still encoding. Your friend's feed still shows yesterday's order because fan-out is backed up. Same product, three different clocks: upload ack, media ready, timeline visible.

"Design Instagram's feed" means: scroll a stream from people you follow, upload photos and video that become visible worldwide, and deliver bytes on mobile without unbounded work per swipe or full-resolution video inlined in every API response.

Most interviews focus on Home (often Stories). Explore is optional unless they widen scope in clarifying questions.

Interviewers grade two layers that a Facebook-style text feed interview may skim:

1. Light metadata — who posted what, order, permissions, candidate post IDs.
2. Heavy media — object storage, transcoding, CDN URLs, adaptive playback, egress cost.

The feed is not one SQL join. It is pointers, async pipelines (fan-out, transcode, notifications via queues or Kafka with idempotent workers and outbox/CDC), and skew: many scroll, few post, a few creators are broadcasters.

Weak answers ship full video in the feed JSON or block post success on ffmpeg. Strong answers cap candidates, bound ranker time, return media_processing states, and say what the UI shows when transcode or CDN lags.

If you remember one thing: Metadata path fast; bytes path async—never block post ack on every encoding ladder.

How to approach

Start the way a user experiences the product, not by drawing every box. Spend the first few minutes turning "design Instagram feed" into a contract you can design against. Talk through clarifying questions out loud, then walk one Reel upload and one feed page before you debate Redis vs Cassandra.

Clarifying questions (say these in the room)

Start with scope—Home vs Stories vs Explore—and how fresh media must be, because those answers change fan-out policy, the read path, and whether you owe an Explore retrieval story.

You: "Is this Home from follows only, or do we also need Explore discovery in the same round?"

Interviewer: "Home feed from follows; Explore can stay out of scope unless we expand."

That keeps you on hybrid fan-out and media pipeline instead of building a full recommender catalog on day one.

You: "For video, does posted mean the HTTP call returns when the blob is durable, or when every transcode ladder rung exists?"

Interviewer: "Users expect fast ack; quality can land async."

Now you owe media_processing states and ABR—not blocking the edge on ffmpeg.

You: "Should I treat creators with tens of millions of followers as a special case?"

Interviewer: "Yes—call out celebrity fan-out explicitly."

You now owe a hybrid policy, not "we fan out to everyone."

You: "Are Stories in scope—24-hour TTL, separate from main feed posts?"

Interviewer: "Yes, mention TTL and expiry; keep ranking shallow unless we push."

Stories need sweepers and server-authoritative expiry—not "posts with a flag."

You: "I want to leave full Explore retrieval, DMs, payments, and end-to-end ML training out of scope so we can focus on upload, hybrid fan-out, home read, and degradation. Does that match what you want?"

Interviewer: "Yes—focus on home feed and media at scale."

Saying out of scope early stops you from spending the round on adjacent products.

Teaching aside: A Facebook feed interview often centers on fan-out and ranking over text-heavy posts. Instagram adds upload resume, transcode queues, variant URLs, and CDN egress as first-class costs. If you only reuse a news-feed diagram without a media lane, you miss half what the room is grading.

Minute pacing (about 45–60 minutes)

Minutes	Focus
0–5	Clarify Home vs Stories vs Explore, post ack vs transcode, celebrities, out of scope
5–12	Capacity: read:write skew, egress, one celebrity fan-out estimate
12–25	Write path: upload → durable post → async transcode + fan-out
25–40	Read path dig: candidates, hydrate, CDN URLs, rank, cursor
40–50	Architectural dig (ack vs ladder) + one production scene
50–60	Questions / deepen one tradeoff

In the room (a fuller opening you can actually say): "I will split metadata and media. Upload lands in object storage; post row becomes durable with processing flags; HTTP success does not wait for every transcode ladder. Fan-out to follower mailboxes runs async with idempotent writes—push for normal authors, merge on read for mega-creators. On scroll I load capped candidate IDs, batch-fetch metadata, map media keys to CDN URLs per variant, filter blocks, rank under a hard timeout with recency fallback, and return an opaque cursor. When transcode lags, the feed still returns posts with processing states—not empty cards."

Out of scope (and why you park these)

Write a short out-of-scope list on the board so the interviewer sees you can bound the problem.

Full Explore / For You retrieval stack. Explore is offline candidate generation plus online rerank over a huge pool—not the same mailbox fan-out as follows. Note "Explore injects candidates elsewhere" without designing the full recommender unless they expand scope.

Direct messages and calling. Real-time messaging is a chat-shaped problem (per-thread order, delivery receipts). Park it unless asked.

Training the ranking model end to end. You need the serving interface: bounded candidates in, scores out, timeout behavior. Feature-store plumbing is a different interview.

GPU beauty filters and AR effects. Client-side or async offline—off the post ack critical path.

Strongly consistent worldwide visibility the millisecond someone posts. Home feed accepts short lag. Promise instant global mailbox sync and you invent a harder problem than the room asked for.

Capacity estimation

Before you draw architecture boxes, use rough numbers so you do not promise impossible per-scroll work or ignore egress.

Dimension	Rough scale	What this means for your design
DAU	100M+ (varies by region)	Home reads dominate; cache and shard aggressively
Feed fetches per DAU	Many sessions; clients reuse data	Not every scroll is a full server reload—but still huge read QPS
Uploads / day	Tens of millions	Video drives storage, transcode CPU, and CDN egress
Read vs write QPS	Far more feed reads than new posts	Optimize read path; async everything on write
Egress	Often the main cost alongside compute	Say data out, not only request counts
Fan-out for 30M followers	30M mailbox writes per post if naive	Must be policy-exempt or partial—queue speed alone does not save you

Worked skew sketch: If one creator posts one Reel and you fan out to thirty million followers, you create on the order of thirty million mailbox inserts for a single tap—before transcode jobs or push notifications. That is why "always fan out" fails, even with a fast queue.

Implications:

• Any design where work per post grows with follower count without a cap breaks for celebrities.
• Transcoding and fan-out should run asynchronously with backpressure when queues grow.
• The feed request should limit how many candidates enter ranking so tail latency (for example p95) stays under control.
• Egress and encoding are first-class costs—not footnotes after QPS.

If you remember one thing: Numbers exist to show skew and bytes—reads dominate, celebrities break naive fan-out, video dominates bandwidth.

High-level architecture

What breaks if you merge everything

One service that stores full post JSON (including video bytes) in every follower mailbox, blocks post success on every transcode ladder, ranks with no timeout, and streams pixels through the feed tier will choke on celebrity traffic, melt the edge when ML slows down, and turn CDN egress into an API bandwidth bill.

What works: thin orchestration, heavy bytes elsewhere

From the phone's perspective, scrolling home means: "give me the next batch of posts after where I left off." The app receives JSON—captions, author fields, and URLs for thumbnails and video—not multi-megabyte files inlined in the response. The device fetches media separately from CDN caches close to the user.

Who owns what:

• API / edge — Auth, rate limits, timeouts to downstreams; upload session initiation.
• Feed service — Orchestrates the read path: candidates, graph checks, hydrate, CDN URL assembly, rank, cursor. Cap work here.
• Mailbox / timeline store — Ordered post IDs per viewer, sharded by viewer_id.
• Post service — Source of truth for captions, author id, timestamps, privacy, media keys, media_processing state.
• Media pipeline — Transcoders, thumbnails, ABR ladders; triggered by queues after upload.
• Object storage + CDN — Blobs and segments; regional edge caching; signed or stable paths.
• Graph service — Follows, blocks, mutes—cached on the feed path; blocks filtered on read.
• Fan-out workers — Consume a queue; append to mailboxes where push policy allows; idempotent (viewer_id, post_id).
• Ranker — Scores a bounded candidate set under a hard time budget.

Celebrity / hybrid on the whiteboard: For authors you do not fully fan out, the feed service merges recent posts by author into the candidate pool. Draw that edge—or you only drew push fan-out and invite the follow-up you deserve.

[ Mobile client ]
    → [ Edge / API ] → [ Feed svc ] → [ Candidate store: mailboxes + optional merge ]
              │              │                    │
              │              │                    └──→ [ Ranker ] (timeout + fallback)
              │              │
              │              ├──→ [ Graph svc ] (follows, blocks)
              │              │
              │              └──→ [ Post svc ] → metadata only
              │                         │
              │                         ├──→ [ Media jobs ] → transcode / thumbs / ABR
              │                         │
              │                         └──→ [ Object store ] → [ CDN ] → (video/image bytes)
              │
              └── upload: [ Upload API ] → object store → enqueue transcode → post row durable
                    fan-out: [ Queue ] → workers → viewer mailboxes (idempotent)

In the interview: Narrate the read path first (latency budget, max candidates, ranker timeout, CDN URL shape). Then one write (durable post → async transcode → async fan-out). Storage brand names come after the story makes sense.

Async half (say it): Transcoding, fan-out, notifications, and search indexing are too slow for the post HTTP handler. Use queues or event streams (PostCreated, TranscodeComplete, FanOutRequested) with at-least-once delivery and idempotent workers. Hand off reliably via transactional outbox or CDC—not "commit then best-effort send" without a lost-message plan.

If you remember one thing: Hot path is IDs → metadata → policy → CDN URLs; pixels never stream through the feed tier.

Core design approaches

Most of the interview is not picking a brand of database—it is walking forks where each choice trades latency, cost, complexity, and failure behavior.

Upload and media processing

Clients upload chunks to object storage; the API creates a durable post row with media_processing: pending. Transcoders build ABR ladders, thumbnails, and poster frames; the row moves to ready or partial as artifacts land.

Decision	If you optimize for…	You usually accept…
Resumable / chunked upload	Flaky mobile networks and large videos	More moving parts (session IDs, part completion, abandoned upload cleanup)
HTTP success when metadata + blob durable	Fast "posted" feeling	Processing states in the API—clients render pending / partial honestly
…every ladder exists before ack	Simpler "ready" mental model	Terrible tail latency and fragile post ACK at scale
Many ladder rungs + renditions	Playback quality and smooth ABR	Storage, transcode CPU, cache cardinality (more URLs to purge)
Fan-out in the upload request	Immediate visibility	Spiky load—most designs enqueue fan-out after the row is durable

Interview line: Separate "can we show something?" (poster frame, low rung) from "is the full product-quality ladder done?"

Feed assembly (push / pull / hybrid)

Policy	When it fits	Cost you accept
Push to follower mailboxes	Long-tail authors (modest follower counts)	Storage and fan-out CPU on publish
Merge / pull recent by author	Celebrities and mega-creators	Extra work on the home read path
Partial fan-out to active users	Middle tier or "online now" optimization	Stale mailboxes for inactive viewers

Push for the long tail. For mega-followed authors, do not materialize every edge. The exact threshold matters less than saying there is a policy and naming the cost driver (queue depth, storage, blast radius).

Stories

Stories share DNA with feed posts (same graph, often same media pipeline) but differ in product rules—most notably short lifetime.

Topic	Typical choice	Tradeoff
Lifetime	Fixed TTL (e.g. 24h)	Sweepers vs lazy delete; server-authoritative expiry
Storage model	Separate Story rows vs same table with type=story	Separate keeps TTL jobs simpler
Seen / replay state	High write volume if you persist every view	Sampling, batching, or eventual sync

Interview line: Stories are time-bounded inventory with expiry semantics—not "feed with a flag."

Ranking

Treat ranking as a contract: bounded candidate IDs in, ordered list out, under latency and safety constraints.

Structural tradeoff: Generating candidates is cheaper than scoring the world. If you conflate them, every feed fetch runs the full recommender—p95 dies. Cap candidates, then rank. Say what happens at the ranker budget: timeout, recency fallback, cached last-good order.

If you remember one thing: Hybrid fan-out plus async media are the Instagram-shaped answers—pure push or sync transcode alone fails.

Data model

At scale there is no single database that wins every access pattern. Name source of truth vs cache vs derived for each arrow.

What lives where

Kind of data	Typical home	Why
Post & user records	Relational DB or document store	ACID-ish create/update; indexes on author_id, created_at
Per-viewer feed candidates	Wide-column, key-value, or tiered mailbox store—sharded by viewer_id	Huge cardinality; append on fan-out; range scan for next page
Social graph	Edges table + heavy cache	Follows drive fan-out; blocks must be correct—read-time checks
Media blobs	Object storage + CDN	Bytes do not belong in OLTP rows
Hot read path	Redis cache, CDN for URLs	Sub-ms repeated metadata; no durability requirement for cache
Async work	Queues / Kafka	Transcode, fan-out, notifications—backpressure and replay

Example schemas (illustrative)

Posts — source of truth for metadata and pointers:

• post_id (PK), author_id, caption, visibility, created_at
• media_processing (pending | partial | ready | failed)
• media_manifest — original key, ladder entries per variant, poster key, thumbnail key

Mailbox / timeline (per viewer)—store IDs, not full posts:

• Key: viewer_id + shard; ordered list: post_id, optional score or inserted_at
• Idempotency on (viewer_id, post_id) for fan-out retries

Graph edges:

• follower_id, followee_id, created_at
• blocker_id, blocked_id, created_at

Stories (often separate table):

• Media pointers plus expires_at, story_kind; seen state optional or batched

Object storage layout

• Prefix by post_id or author_id/post_id for lifecycle deletes.
• Originals vs derived artifacts in separate keys so transcode can retry per artifact.
• CDN URLs signed or stable; purge on bad encode or takedown.

Redis and caches (trimmed)

Redis is fast RAM with TTL—not durable post storage unless you are explicit about sync.

Strong fits: cache-aside for hot post metadata and profiles; rate limiting; small bounded per-user tail caches for merge; coordination keys for fan-out idempotency.

Risky fits: entire mailbox for every user in Redis only—memory explodes; recovery after failover is hard. Production systems often use wide-column mailboxes with Redis as a cache layer on top.

Interview land: Postgres (or similar) holds authoritative posts; mailbox storage holds IDs at scale; object storage holds media; Redis caches hot reads and powers rate limits.

Detailed design

Walk one post from “user tapped Share” to “friends can scroll it,” then one home-feed page load. Media makes Instagram different from a text-heavy social feed: the path that stores captions and pointers must stay fast, while video encoding and CDN delivery move on their own clock.

Write path (post + media)

On a phone with spotty LTE, uploading a long Reel as one unbroken HTTP request often fails mid-way. So the client usually requests upload URLs (or chunked upload sessions), pushes the original file into object storage, and only then asks the Post API to create the story metadata. That Post API call records who posted, the caption, visibility, timestamps, and the object keys—plus a processing flag such as media_processing: pending.

Success on that API means “we safely stored what we need to remember this post,” not “every quality level of the video already exists.” Returning only after ffmpeg finishes every bitrate would make post time track the worst encoding queue in the building. Instead, workers transcode thumbnails, poster frames, and adaptive bitrate ladders in the background, and they update the media manifest as each artifact lands. The phone can show a poster frame and start playback on a lower rung while higher rungs are still cooking.

When product rules say the post may appear in followers’ timelines, fan-out workers append the post id into follower mailboxes (idempotently, so retries do not duplicate). For mega-creators you skip full fan-out and rely on merge-on-read—same hybrid idea as Facebook Feed, with even more pressure because each item also references large media. Search index updates, notifications, and Story-specific fan-out should trail this path via an outbox or event stream so a slow notifier cannot block “posted.”

Read path (home feed)

When someone opens Home, the feed service first knows who is scrolling and which product mode they are in (following vs “for you,” if that is in scope). It loads a bounded set of candidate post ids from that viewer’s mailbox, plus any celebrity merge. Bounding matters: ranking cannot score “everything everyone I follow ever posted” on every swipe.

Next it batch-loads post metadata and author cards, then filters blocks and hard policy at read time so a blocked account does not slip through a stale mailbox entry. For each surviving post it chooses which media URLs to return—thumbnail, poster, HLS manifest—based on the client’s capability and the current media_processing state. Ranking runs only on that capped list, with a hard time budget; if the model is slow, fall back to recency or a cached order rather than spinning an empty feed. The response stays a small JSON page of ids, captions, and CDN pointers; the client fetches video bytes from the CDN separately. An opaque cursor lets the next swipe continue without offset pagination lies when order moves.

In the room (spoken): “Upload to object storage, durable post row with processing pending, ack before full ladders exist. Async transcode and hybrid fan-out. On read, cap candidates, hydrate metadata, map to CDN variants, rank under a timeout, return a cursor—never put video bytes in the feed JSON.”

Architectural dig: one Reel post (ack vs ladder) and one celebrity publish

High-level boxes showed ownership. This dig is how posting stays fast when ffmpeg is slow, and how scroll stays cheap when a creator has thirty million followers.

1) Post ack: metadata durable, ladders async

Responsibility: Post service owns durability and media_processing state. Media workers own transcode artifacts. Feed service must not wait on every ladder before the client gets success.

1. Client finishes chunked upload to object storage.
2. Post API persists row with media keys and media_processing: pending.
3. Return 201 with post_id and processing state; client shows poster frame or optimistic UI.
4. Enqueue transcode jobs; workers emit TranscodeComplete per variant; row moves to partial then ready.
5. Fan-out enqueues when visibility rules say so—often after basic thumb/poster exists, not after every rung.

2) Ack vs full transcode ladder (bad vs great)

Bad design: block HTTP on every encoding rung

User on 3G posts a 60s Reel. API waits for ffmpeg to finish all bitrates before returning. Post latency spikes to minutes; gateways time out; retries duplicate work; users think the app is broken even when upload succeeded.

Upload done ──► API waits for full ladder ──► timeout / retry storm
                     │
                     └── user sees spinner; fan-out never starts

Great design: ack on durable metadata + blob; ladders land async

Upload and post row commit; API returns immediately with media_processing: pending. Client plays lowest available rung via ABR while higher bitrates land. Feed returns the post with processing state—never an empty timeline.

Figure: Success on post is not success on every bitrate. The API contract must expose processing state; the client and CDN handle progressive quality.

Interview land: "I never block post ack on ffmpeg. I do block fan-out on nothing—metadata must be durable first."

3) Celebrity blast radius (bad vs great)

Bad design: always push to every follower

A creator with thirty million followers hits Post. Workers attempt thirty million mailbox writes. Fan-out queue ages; friends' feeds look stale for minutes while create-post dashboards stay green.

Post 201 ──► enqueue fan-out ──► 30M mailbox writes
                │
                └── queue age ↑ while API success looks fine

Great design: hybrid policy by follower tier

Classify the author. Long-tail: push to mailboxes. Mega-followed: store in an author-indexed recent list and merge on read into each viewer's candidate set (optionally push only to recently active viewers).

Long-tail author  → push post_id to follower mailboxes
Mega-creator        → append to author tail; feed read merges into candidates

Scale inside the box: Monitor queue age, writes per shard, and time-to-first-visibility—not only create 201 rate.

Failure inside the box: If workers die mid-fan-out, idempotent retries finish the long tail; celebrity merge still lets actives see content from the author store.

4) Read: bounded candidates under a rank budget

1. Authn viewer; load block/mute set (strict freshness for blocks).
2. Read mailbox head + merge celebrity tails → cap candidates (hundreds, not unbounded).
3. Batch-get posts + authors; drop tombstones and blocked authors.
4. Map media keys to CDN URLs per variant; include media_processing in each item.
5. Call ranker with deadline T ms; on timeout use recency or last known order.
6. Return page of size K + opaque next_cursor.

5) Numbered happy path (put it together)

1. Upload → durable blob + post row → 201 with pending processing.
2. Transcode jobs land variants async; fan-out (or celebrity skip) → mailbox IDs where policy allows.
3. Home GET → candidates → hydrate → CDN URLs → filter → rank or fallback → cursor.
4. Client fetches bytes from CDN; ABR upgrades as ladders complete.

If you remember one thing: Post ack, media ready, and timeline visible are three different SLOs.

Key challenges

Challenge	What users see if it goes wrong	What to say in the interview
Transcode lag	"Posted" but Reel spins on processing for hours	Ack on metadata; media_processing states; priority lanes for transcode
Celebrity fan-out	Friends' feeds stuck while posting still "works"	Hybrid policy; queue age alerts; merge on read
CDN / egress hot key	One viral Reel stutters globally	Variants per resolution; origin shield; singleflight; watch egress per key
Moving ranks	Duplicate posts or gaps while scrolling	Opaque cursors; admit best-effort dup/skip
Stories expiry	Ghost stories or missing new ones	Server-authoritative TTL; sweeper metrics; CDN manifest TTL
Like hot keys	Feed fine; like counts wrong or slow	Shard counters per post_id; cache reads; rate-limit writes

If you remember one thing: Video + skew dominate cost—fan-out, transcode, and egress are not afterthoughts.

Scaling the system

Shard mailboxes by viewer_id. Shard post metadata by post_id or author-time. CDN is how you scale bytes—not bigger feed JSON.

For hot posts, use request coalescing (singleflight) and TTL jitter on metadata caches—thundering herd on a viral Reel ID is a classic failure mode.

Multi-region: Graph and media replication lag is real; eventual visibility across regions is more honest than strong global consistency for every mailbox row.

Cap pull-merge fan-in when someone follows thousands of accounts—otherwise hybrid still burns CPU on read.

If you remember one thing: CDN scales bytes; mailboxes scale read QPS; transcode fleet scales encode backlog.

Failure handling

Scenario	What the user sees	What to build
Ranker slow or down	Spinners or empty home	Hard timeout; recency fallback; circuit breaker; feature-flag heavy models
Transcode backlog	Post succeeded; Reel stays on processing	Serve poster / low rung; alert on oldest job age; priority lanes
CDN / origin hot spot	One viral Reel stutters; origin errors	Origin shield; singleflight; regional failover; monitor origin per key
Fan-out backlog	Post succeeded; friends see stale home	Alert on queue age; merge-on-read; reads succeed from stale mailboxes + merge
Graph service degraded	Blocked user appears briefly	Circuit break; filter blocks at read; fail closed where abuse risk is high

If you remember one thing: Never return an empty feed when transcode lags—return posts with processing media states.

API design

Illustrative REST surface (GraphQL is fine if you discuss batching and N+1—feeds amplify bad graphs).

POST   /v1/media/uploads              # or /uploads:init + chunked parts
POST   /v1/posts
GET    /v1/feed
GET    /v1/posts/{post_id}
GET    /v1/stories/reel
POST   /v1/users/{user_id}/follow
DELETE /v1/users/{user_id}/follow
POST   /v1/posts/{post_id}/like
GET    /v1/posts/{post_id}/comments

Home feed — GET /v1/feed

Query param	Role
cursor	Opaque continuation; may embed ranker state
limit	Page size (cap server-side, e.g. ≤ 20–50)
mode	e.g. following vs for_you if product has both
session_id	Experiments / snapshot behavior (optional)

Response sketch: { "items": [ Post ], "next_cursor": "...", "has_more": true }. Each Post includes media with status (ready/processing/partial), urls or playback descriptors per variant, counts, author summary.

Errors: 401 / 429 with Retry-After; avoid leaking blocked user existence if policy requires.

Upload — POST /v1/media/uploads

• Return upload session id and part URLs or signed PUT targets.
• Idempotency-Key on POST /v1/posts so flaky networks do not duplicate posts.

Stories — read

Separate resource with TTL hints in response; server is source of truth for ordering, privacy, and expiry.

Cross-cutting

• Rate limits on upload and create to fight abuse.
• Version mobile clients; backward-compatible field adds for media_processing states.

In the interview: One minute on idempotent post create, cursor semantics, media_processing in responses, and why likes do not own feed p95.

Observability (what you measure)

Signal	Why it matters
Post create success vs transcode oldest job age	Explains "posted but Reel still processing"
Post create vs fan-out queue age	Explains "posted but friends do not see it"
Time-to-first-visibility (sampled)	Product freshness SLO
Ranker p99 + timeout / fallback rate	Separates rank health from feed availability
Candidates scored per request	Catches unbounded merge
CDN origin QPS / egress per hot object key	Viral Reel stampede and cost
media_processing pending rate in feed responses	Honest degradation visibility
Empty feed / 5xx rate	User-visible outages

Green create rates alone will lie while timelines feel stale or Reels spin on processing.

Security and abuse

Authenticate every feed and create call. Authorize visibility (public, followers-only, blocked) on read, not only at fan-out time. Rate-limit upload, post create, follow, and like to slow spam and bot farms. Signed upload URLs with short TTL and content-type constraints reduce arbitrary blob abuse. Treat ranking and moderation logs carefully—do not dump private captions into unrestricted analytics. Report flows and integrity pipelines (fake engagement, coordinated amplification) belong beside the feed, not as synchronous mailbox rewrites.

Cost awareness

Every pushed mailbox row costs storage and write IOPS until trimmed. Celebrity push-to-all is a cloud bill incident. Transcode CPU and CDN egress often dominate Instagram-shaped systems—video bytes are not free like text metadata. Many ABR ladder rungs improve playback but multiply storage, encode cost, and cache cardinality. Caching post metadata reduces post-store load but needs stampede controls on viral IDs. Choose explicitly whether you spend money on precompute storage, read-time merge CPU, or encode/egress quality.

Production scenes

Textbooks teach boxes and arrows. On-call teaches this: post create can look healthy while Reels spin on processing and feeds feel stuck in yesterday.

Scene 1: Transcode backlog while posts "succeed"

The moment: User taps Share on a Reel. App shows "posted." Playback stays on a poster frame or low quality for hours after an incident.

The trap: Transcode capacity is finite. Metadata and fan-out can run while ladders lag. Dashboards that only chart create 201 hide oldest job age.

What to build: Alert oldest transcode job age; priority lanes for user-visible posts; serve low rungs first; honest media_processing in API; feature flag to relax non-critical encodes during incidents.

Scene 2: Fan-out backlog while posts "succeed"

The moment: A live event or creator posts. Create still works. Friends refresh and still see yesterday's top of feed.

The trap: Fan-out is async. Workers append at finite speed. Time-to-visible is a different SLO than post ack.

What to build: Alert on queue age and per-shard backlog. Hybrid merge on read while queue drains. Priority lanes for recent posts vs backlog drain.

Scene 3: Ranker deploy hangs the feed

The moment: After a ranking release, homes spin; unrelated routes feel slow.

The trap: Ranker sits on the hot path without isolation. Missing timeout fills the edge thread pool.

What to build: Hard rank timeout; circuit breaker; recency fallback; kill switch for heavy models. Separate SLOs: feed loads vs perfect rank.

Scene 4: CDN stampede on one viral Reel object

The moment: One viral Reel; feed still loads but p99 spikes; origin errors while average cache hit ratio looks fine.

The trap: Every scroll multi-gets the same post_id and CDN object key. Aligned TTL expiry creates a miss herd; egress spikes.

What to build: Singleflight per key; origin shield; TTL jitter; stale-while-revalidate; watch origin QPS and egress on one key, not only cluster CPU.

In the interview: Pick one scene. Tell what users felt, which metric lied, and what you would ship next.

Bottlenecks and tradeoffs

Encoding cost vs playback quality

The tension — More ABR ladders and prefetch feel great on Wi‑Fi; each variant is storage + transcode CPU + CDN cache cardinality.

What breaks — Transcode queue backlog; users see processing forever during incidents.

What teams do — Priority lanes; serve low rungs first; shed non-critical encodes; honest media_processing states in API.

Say in the interview — Separate "can show something" from "full ladder done."

Push fan-out vs read merge

The tension — Push makes scroll cheap; celebrities make writes impossible.

What breaks — Queue depth and storage blast radius on one post.

What teams do — Hybrid policy by follower count; merge author tail on read; cap candidates.

Say in the interview — Name who you fan out to—not "Kafka."

Metadata path vs bytes path

The tension — Inlining media in feed JSON feels simple; it destroys latency and egress economics.

What breaks — API bandwidth bills; p95 blows up on large responses; CDN never gets a cache hit on personalized JSON.

What teams do — Feed returns URLs and processing state; client fetches variants from CDN; ABR handles partial ladders.

Say in the interview — IDs → URLs on the hot path; pixels on the CDN.

If you remember one thing: Instagram tradeoffs are bytes + skew—not only feed algorithm labels.

Interview tips

You have the full design in mind now. These six back-and-forths are how interviewers test whether you can defend it under pressure. Each one starts with a tempting shortcut, an interview push, and a clear answer.

Post ack vs video ready

You might say: "We wait until transcoding finishes before returning success."

Interview Push: "User on 3G posts a 60s Reel—what happens to post latency?"

Land here: Return success when metadata and blob upload are durable; expose media_processing: pending in the API; client shows poster frame and ABR while ladders land async. Blocking HTTP on ffmpeg is a scale mistake. Say what the feed returns while processing—never an empty timeline.

Fan-out to everyone

You might say: "Every post fans out to all followers immediately."

Interview Push: "Creator with 30M followers—how many mailbox writes?"

Land here: On the order of thirty million mailbox inserts for one click if naive. Use hybrid: push for long tail; merge on read or author-indexed tail for head; partial fan-out to active users only. Name queue depth and storage blast radius—not only "we use Kafka."

Feed as one join

You might say: "SELECT posts JOIN follows ORDER BY time."

Interview Push: "Someone follows 5,000 accounts—cost per swipe?"

Land here: Candidate IDs from mailboxes + optional merge; multi-get post rows; no giant join on hot path. Cap candidates before rank. Feed JSON carries pointers and CDN URLs—not video bytes.

CDN as a label

You might say: "We use a CDN."

Interview Push: "Viral Reel—what object key melts, and what do you purge?"

Land here: Variants per resolution/ladder; signed URLs; origin shield; hot key on one object id—monitor egress and origin CPU, not only API QPS. Purge or invalidate per variant on bad encode or takedown.

Ranking on the critical path

You might say: "Home For You runs the full model synchronously."

Interview Push: "Feature store slow—does scroll stop?"

Land here: Bounded candidates → rank with hard timeout → recency fallback. Separate candidate generation from scoring. Perfect rank that empties the feed is a product failure.

Stories bolted on

You might say: "Stories are just posts with a flag."

Interview Push: "What happens at 24 hours?"

Land here: TTL, sweepers, often separate storage/API; different ranking hooks; server-authoritative expiry—not only client hide. Align CDN manifest TTL with metadata expiry.

Red flags to avoid without correcting yourself: "Block post on transcode," "fan out to everyone," "full video in feed JSON," "rank with no timeout," "Stories are only a flag."

After each interview push, close with one real part of the design you would build—processing states, hybrid threshold, idempotent (viewer_id, post_id), ranker timeout, or CDN variant URLs—not "we'll scale it later."

What should stick

After reading this guide, you should be able to explain:

1. Two layers — Fast metadata path; heavy media async (upload → transcode → CDN).
2. Hybrid fan-out — Push where cheap; merge for mega-creators; bounded candidates before rank.
3. Post ack ≠ all bitrates — media_processing states; ABR and poster frames first.
4. Hot path = IDs → URLs — Bytes ride CDN; feed JSON stays small.
5. Degrade honestly — Ranker timeout, transcode backlog, CDN miss—processing states beat blank screens.
6. Instagram vs Facebook feed — Same fan-out and ranking bones; media upload, transcode, and egress are the extra half.

Tell it in the room: "Upload to object storage, durable post row with processing flags, async transcode and idempotent fan-out to mailboxes where policy allows. Read: capped candidate IDs, multi-get metadata, CDN URLs per variant, rank with timeout, cursor. Celebrity: merge on read. When transcode lags, show processing; when ranker lags, recency fallback; Stories need TTL and sweepers."

Key Takeaways

• Clarify first — Home vs Stories vs Explore, post ack vs full ladder, celebrity policy, out of scope.
• Metadata fast, bytes async — Durable post row before every transcode rung; expose processing state.
• Hybrid fan-out — Push the long tail; merge mega-creators on read.
• Split storage — Mailboxes store IDs; posts and media live elsewhere.
• Bound the read path — Cap candidates; hydrate in batch; CDN URLs per variant; rank with a deadline.
• Media-heavy cost — Transcode CPU and CDN egress are first-class—not footnotes.
• Degrade ranking and media honestly — Recency fallback and processing states beat empty spinners.
• Operate queue age, transcode age, and hot keys — Create 201 can look fine while Reels spin and feeds feel stale.

Related Topics

• Facebook Feed System Design — Text-heavy feed contrast: same hybrid fan-out and ranking bones without the media pipeline depth
• WhatsApp System Design — Messaging contrast: per-thread order vs feed fan-out and CDN-backed media pointers
• Distributed Cache System Design — Hot post metadata, stampede, and TTL jitter on viral IDs
• CDN System Design — Edge caching, origin shield, and egress when every Reel references CDN URLs
• Rate Limiter System Design — Protecting upload, create, and follow from abuse

Quick revision notes

Use this as a last look before a mock interview, not as a substitute for the full guide.

Clarify first. Agree Home scope, whether Stories are in, that post ack is metadata-durable not ladder-complete, that celebrities need hybrid policy, and that Explore stays out unless they expand. Skipping this often means you draw Kafka with no bytes story.

Two layers. Metadata path is fast—IDs, captions, processing flags, CDN URLs. Bytes path is async—upload, transcode, fan-out, notifications. Never block post success on every ffmpeg rung.

Pay on publish or on scroll—pick by follower tier. Push mailbox IDs for normal authors. For mega-creators, merge recent posts on read so one publish does not become tens of millions of writes.

Keep the timeline thin. Mailboxes store ordered IDs. Post bodies hydrate in batch. Ranking scores a bounded set under a timeout. Client fetches media from CDN with ABR while ladders complete.

Paginate and degrade honestly. Opaque cursors beat offsets when ranks move. Blocks filter on read. Transcode backlog returns processing states—not empty feeds.

Operate the lag. Alert on transcode oldest job age, fan-out queue age, ranker timeouts, and hot CDN object keys—because green create rates and angry users can coexist.

Practice this path: Instagram Feed System Design.