WhatsApp System Design

Introduction

You already know the headline from above: billions expect private chat that feels instant, with delivery ticks they trust and encryption that keeps the server from reading message text. This section explains what that means in practice and what interviewers listen for.

A messaging app lets people send text, photos, and receipts in threads that feel live. That sounds simple. The hard part is everything that happens on a bad network, in a forty-person family group, or when one viral channel sends hundreds of messages per second—while phones retry sends and still expect correct order.

On a quiet 1:1 chat, the system behaves like a small API: encrypt, save, notify. At scale, the pattern changes. Millions of phones keep long-lived connections open. Groups force you to choose between copying every message to every inbox or reading from one shared log. Video uploads cannot block the same path as a one-line "running late." If messages show up twice, out of order, or plain text lands in server logs, users lose trust fast.

To design this well, keep three ideas separate in your head. They are related, but they are not the same problem.

First: delivery and order per chat. Mobile networks retry. The same "hello" can arrive twice unless the server recognizes a duplicate. Order cannot rely on each phone's clock—time zones and skew disagree. The server assigns a rising number per conversation (seq) and dedupes with a client_msg_id from the sender.

Second: privacy (end-to-end encryption). The server stores and routes scrambled bytes (ciphertext), not readable chat text. That choice limits what push notifications, search, and support tools can do on the server—you must name those tradeoffs aloud.

Third: two speeds of data. Short control messages (text, receipts, "typing…") need a fast lane. Photos and video need a separate upload path (object storage + CDN) so a huge file never blocks checkmarks on a one-line message.

In interviews, weak answers draw one box labeled "message queue" or log readable messages "for debugging." Strong answers walk through one send: encrypt → save to inbox → notify online peer on the socket or wake offline peer with generic push → explain group fan-out with a stated max size.

How to approach

Start the way a user would experience the product, not by listing every service name. Spend the first few minutes turning a vague “design WhatsApp” prompt into a contract you can design against. Talk through clarifying questions out loud, then walk one text send end to end before you name WebSockets or object storage.

Clarifying questions (say these in the room)

Start with group size and encryption depth, because every later choice (inbox fan-out, push text, search, support tooling) hangs off those two answers.

You: “What is the maximum group size—a dozen family members, a few hundred teammates, or a stadium channel? That picks whether we copy each message into every inbox or store once and let members read with a cursor.”

Interviewer: “Assume up to a few hundred for normal groups; call out what changes if someone wants fifty thousand.”

That answer tells you copy-on-send is fine for small groups and that you must name a shared-log or announcement model above your cap.

You: “How deep should I go on cryptography—high-level end-to-end encryption, or a full Double Ratchet walkthrough?”

Interviewer: “High-level E2E is enough unless I push.”

Now you know to spend minutes on inbox, order, and fan-out—not twenty minutes on crypto math—while still saying the server stores ciphertext only.

You: “Do users have multiple devices that must all show the same thread, and do we need read receipts that tell who has opened a message in a large group?”

Interviewer: “Yes to multi-device; pick a product bar for receipts in big groups.”

Multi-device means encrypting for each device and syncing cursors; large-group receipts need throttling or aggregates so one “read” event per member per message does not melt writes.

You: “When the phone is offline, is the delivery promise ‘saved on the server for later sync,’ or ‘push must succeed’?”

Interviewer: “Inbox on the server is the promise; push is best-effort.”

That keeps you from treating Apple or Google delivery as proof the human saw the chat.

You: “I want to leave full cryptographic protocol math, a complete payments/commerce product, and building the national SMS OTP network out of scope so we can focus on send, order, offline sync, groups, and E2E limits. Does that match what you want?”

Interviewer: “Yes—focus on messaging reliability and privacy constraints.”

Saying out of scope early stops you from burning the round on adjacent products.

Minute pacing (about 45–60 minutes)

Minutes	Focus
0–5	Clarify group size, E2E depth, multi-device, offline promise, out of scope
5–12	Capacity + two lanes (fast text vs media) + connection sketch
12–30	One send dig: client_msg_id, seq, inbox commit, socket vs push
30–40	Groups, hot partitions, receipts at scale
40–50	One production scene + failure modes
50–60	Questions / deepen one tradeoff

In the room (a fuller opening you can actually say): “I will separate a fast signaling path for text, receipts, and presence from a truck lane for photos and video. I will walk one text send: encrypt on the phone, send a unique client_msg_id, have the server assign a rising seq per chat, save ciphertext to the inbox before acknowledging the sender, then notify an online peer on the live socket or wake an offline peer with a generic push. For groups I will state a max size—copy-on-send below it, shared log with cursors or announcement mode above it. End-to-end encryption means the server routes ciphertext and metadata only, so lock-screen push usually cannot quote the message body.”

Out of scope (and why you park these)

Write a short out-of-scope list on the board so the interviewer sees you can bound the problem. Explain each line in a sentence, not only as a bullet label.

Full cryptographic protocol design (Double Ratchet math, session setup proofs). You need the system consequences of E2E—ciphertext storage, generic push, limited server search—not a cryptography lecture, unless they explicitly ask for more depth.

Building the SMS / carrier OTP identity network itself. Account signup matters, but verifying phone numbers is a separate identity path. Mention rate-limited OTP and fraud in one breath, then return to inbox and connections.

A full payments, status-stories, or broadcast-channel product catalog. Adjacent surfaces steal whiteboard time from the hard path: ordered durable delivery under retries and group fan-out.

Perfect global linear order across every chat on earth. Chat needs per-conversation order. Cross-region strict linearizability for all threads fights physics and interview pacing—say what you guarantee per chat and what failover may briefly relax.

Capacity estimation

Before you draw architecture boxes, use rough numbers to stay realistic. You do not need perfect statistics. You need a sense of scale so you do not promise something impossible—like copying one "hello" into fifty thousand inboxes in a single instant.

On most days, message volume is spread across millions of chats. During a live event or viral group, one conversation can drive a huge fraction of writes to a single database partition. If you design only for average QPS, evening peak or one stadium channel will surprise you.

Dimension	Rough scale	What this means for your design
Peak messages	~600K per second (order of magnitude)	Shard by chat id or user inbox; keep each send transaction short
Storage	Petabytes of history	Archive old threads; support delete-for-everyone; compress cold data
Live connections	~100M+ open sockets	Regional connection clusters; tune for many idle sockets, not only CPU
Media volume	Much larger than text	Object storage + CDN; fast path carries pointers only

Three practical lessons follow from this table:

• Do not put full video uploads on the same urgent path as text and read receipts.
• Do not treat a stadium-sized group like a twelve-person family chat—product must set max group size or switch storage models.
• When SendMessage returns success to the sender, that should mean the message row is saved in the inbox—not that Apple or Google already showed a notification.

High-level architecture

What breaks if you merge everything

One API that takes full video uploads and read receipts on the same urgent path will choke at peak. Handshakes, database writes, and push notifications all fight for the same few milliseconds. If you also log readable messages for “debugging,” you fail the encryption question before scale even matters.

What works: two lanes

Picture two roads:

• Fast lane — short messages: text, who read what, “user is online,” connection events.
• Truck lane — photos and video: upload to file storage, download via CDN.

The server moves scrambled bytes. It does not read the chat content.

How it fits together:

Phones keep a long-lived connection (WebSocket, MQTT, or similar over TLS) to connection servers in each region. The chat service receives encrypted text + small metadata, assigns msg_id and a sequence number per chat (seq), and saves to a database split across many machines (by chat id or by each user’s inbox). If the friend is online, send over the live connection. If offline, save to their inbox first, then send APNs/FCM push. For media, the phone uploads with a pre-signed URL; the fast lane only carries a file id.

Who does what:

• Connection servers — Keep sockets alive, heartbeats, limit queue size per user so one slow phone does not blow memory.
• Chat / inbox service — Owns message order (seq), saves messages reliably, tracks sent / delivered / read.
• Object store + CDN — Stores files; supports resume if upload breaks.
• Push service — Maps user → phone tokens; respects notification settings.
• Keys (sketch) — Stores public keys per device. Never log readable messages.

Simple version: text uses the fast lane; photos and video use the truck lane.

[ Mobile client ]
      |
      | 1) TLS + long-lived conn (signaling)
      v
[ Edge / Connection cluster ] ──► [ Chat / Inbox service ]
      |                                    |
      |                                    | persist ciphertext + metadata
      |                                    v
      |                             [ Message store shards ]
      |                                    |
      | 2) notify online peers             +--> [ Push ] --> APNs/FCM (offline)
      |
[ Same client or peer ] ◄── ack / delivery / read events

  Media path (async, large):
  Client --presigned PUT--> [ Object storage ] --> CDN URL in message body (encrypted)

Figure: Fast lane for text (blue), delivery to online users (green), and separate upload path for media (orange). The server only stores scrambled message bytes.

In the interview, three short phrases help you sound precise. Each one answers a follow-up interviewers often ask:

• Inbox before ack. When SendMessage succeeds, you are promising the ciphertext row exists in durable inbox storage—the sender can show one gray tick. Do not describe delivery as "done" only because a push worker queued a job or because Apple accepted a notification.

• Push is a hint, not the message. Under end-to-end encryption, the notification often says only "New message." The phone must open the app (or use a live socket) to fetch and decrypt. Push failure does not mean the message was lost if the inbox row exists.

• Fast lane vs truck lane. Text, receipts, and presence use the low-latency signaling path. Photos and video upload separately; the chat message body carries a media_id pointer. Never block checkmarks on a multi-gigabyte upload.

Core design approaches

1:1 messaging

Alice sends one line to Bob. Her phone encrypts, creates a client_msg_id (so retries do not duplicate), and calls SendMessage. The server skips duplicates, assigns seq, saves to Bob’s inbox, then either sends on Bob’s open connection or sends a simple push (“new message”). Bob’s phone decrypts and sends delivered / read when the UI allows.

Order is per chat, not one clock for the whole app.

Group messaging

A group of twelve can copy each message into twelve inboxes and still feel fine. A stadium with fifty thousand people cannot—one "hello" would mean fifty thousand writes unless you change the model.

These models trade write cost against read cost—say your max group size and which you pick above it:

Model	Plain idea	Good when	Watch out
Copy on send	One database row per member inbox per message	Small groups (family, team chat)	Write cost grows with member count
Shared group log	Store once; members fetch with cursor	Large groups, broadcast channels	Read cost; hot partition on viral group
Announcement mode	Only admins post; members mostly read	Stadium-scale updates	Different product semantics than free chat

Figure: Group size is not a footnote—it picks your storage model and your hot-partition risk.

Receipts

Delivered (two gray ticks) usually means the server placed the message on the recipient's live connection or in their inbox. Read (two blue ticks) means the recipient's app reported they opened the thread—the server forwards that signal but does not independently know the user read with their eyes.

In a twelve-person family group, per-member read events are fine. In a five-hundred-person group, one read event per member per message explodes write load—throttle, batch, show "read by 42" aggregates, or disable granular receipts. Say which product bar you pick in the interview.

Data model

Think in three stores: the message / inbox plane (hot path), the media plane (pointers only on the fast path), and light presence / device state that must stay cheap.

Messages and inboxes

Field / concept	Why it exists
conversation_id	Shard and sequence key for the thread
msg_id	Server-assigned stable identity returned to the client
seq	Rising integer per conversation—the order users should render
client_msg_id	Sender-generated id so retries map to one bubble
sender_id, recipient / membership	Routing and access checks
ciphertext + small metadata	Body the server cannot read under E2E; size, media pointer, timestamps
delivery_state	Sent / delivered / read signals the product surfaces as ticks

For copy-on-send groups, you materialize (or fan out) rows toward each member inbox. For a shared group log, you store one sequence of messages for the group and keep a cursor / last_seq` per member so each phone knows what to fetch next.

Devices, keys, and push

Concept	Where it lives	Note
Device public keys	Key directory / identity service	Used so senders can encrypt for each device
Push tokens (APNs/FCM)	Push mapping store	Delete dead tokens; never place plaintext bodies in the map
Media blobs	Object storage + CDN	Encrypted client-side when product requires it; chat row holds media_id

Interview land: Draw conversation_id → (seq, msg_id, client_msg_id, ciphertext) and say which secondary indexes you need for sync-by-cursor—not a giant “messages” table with no key story.

Detailed design

This section follows one ordinary text message from Alice's phone to Bob's phone. The goal is to make the send path understandable before we dig into the retry race, so keep the words tied to what each side experiences: Alice taps send, the server saves a durable row, Bob catches up from the inbox, and the live socket is only the fast way to hear about work that is already safe.

When Alice taps Send, her phone encrypts the message body before it leaves the device. The server receives ciphertext, which means scrambled bytes it can route and store but not read as chat text. Alice's phone also attaches a client_msg_id, a unique id for that one tap. This id matters because mobile networks often lose responses. If the first request succeeds on the server but the phone never hears back, the phone can retry with the same client_msg_id and the server can answer, "I already saved this exact send," instead of creating a second bubble.

The chat service first checks whether (conversation_id, client_msg_id) already exists. If it does, the service returns the same msg_id and seq it created the first time. If it does not, the service creates a stable msg_id for the message and assigns the next seq, a rising number inside that one conversation. The seq is the order the UI should render. It is deliberately not the sender's phone clock, because clocks can be wrong and packets can arrive in a different order from the one the user expects.

After identity and ordering are settled, the service saves the ciphertext and small routing metadata to inbox storage. That save is the delivery promise. If SendMessage returns success before the inbox row is durable, Alice may see a sent tick for a message the system can still lose. Once the row commits, the server can acknowledge Alice and move into notification work: online devices receive a frame on their live connection, while offline devices receive a generic push notification that only says something like "New message." Under true end-to-end encryption, the push provider and the server should not carry the readable body.

Bob's app has two ways to learn about the row. If the app has an open socket, the connection server can deliver the new ciphertext immediately. If the phone is asleep, out of network, or the push never appears because of battery settings or notification permissions, Bob still catches up later by calling SyncMessages(cursor). The cursor is Bob's last known position in the conversation or inbox. The server returns ciphertext rows after that cursor, Bob's phone decrypts them locally, and the UI renders them in seq order.

Receipts follow the same "save the truth, notify as a hint" pattern. A delivered receipt usually means the server placed the message on a device connection or made it available in the inbox. A read receipt is different: Bob's app reports that the thread was opened, and the server forwards that signal according to product rules. In large groups, you should throttle or aggregate those receipts because one read event per person per message can become more expensive than the message itself.

Media uses a separate path. Alice uploads an encrypted photo or video to object storage through a resumable upload, often using a pre-signed URL. Only after the upload has a stable media_id does the chat message reference that media pointer on the fast path. A huge video should not hold the same transaction that assigns a text message seq, and a broken media upload should not block every receipt in the conversation.

In the room (spoken recap): "One send works like this: the client encrypts, reuses a stable client_msg_id on retry, the server dedupes that id, assigns one per-chat seq, saves ciphertext to the inbox before acknowledging, then notifies online devices on sockets or offline devices with generic push. The inbox is the truth; push and sockets are delivery hints. Bob catches up with SyncMessages(cursor) and renders by server seq, not phone timestamp."

Architectural dig

One SendMessage under retries

High-level boxes told you where traffic goes. This dig is how one send stays correct when the elevator kills TCP mid-flight and two phones disagree about “what time it is.”

1) Connection and session ownership

Responsibility: Connection servers own live sockets, heartbeats, and per-user outbound queues. The chat service owns durable order and inbox rows.

Interview land: Sticky routing helps session locality, but durability never lives only in connection-server memory—reconnect must resume from an inbox cursor.

2) Race on retries (bad vs great)

Imagine Alice taps Send in an elevator. The request reaches the server and commits, but the response never returns. Alice’s app times out and sends the same text again.

Bad design: new id every attempt, order by phone clock

Each retry invents a fresh client identity (or none). The server treats each attempt as a new message and sorts bubbles by the phone’s wall clock. Users see two identical bubbles, or a later retry that leaps ahead because clocks are skewed across regions.

Attempt 1: commit message A (wall 12:00:01)
Response lost ↓
Attempt 2: commit message B (wall 12:00:02)  ← duplicate bubble
UI sorts by wall clock → wrong or doubled story

Great design: stable client_msg_id + server seq

The client reuses one client_msg_id for that tap. The server looks up (conversation_id, client_msg_id):

• If found → return the same msg_id and seq (HTTP 409 or success with identical body—say which).
• If not → allocate the next seq, persist ciphertext, then ack.

Rendering sorts by seq, then msg_id on ties—not by wall clock.

Attempt 1: client_msg_id=X → seq=17 commit → response lost
Attempt 2: client_msg_id=X → same seq=17 returned
UI shows one bubble in seq order

Figure: At-least-once networks need idempotent sends. The server seq—not the phone clock—owns order.

Why this shape: Mobile networks are at-least-once. At-most-once delivery “to keep the design simple” loses messages users care about. Wall clocks fight time zones and NTP skew.

Scale inside the box: Idempotency keys need a TTL or retention window per chat so the lookup table does not grow forever. Sequence allocation for one hot conversation_id is a natural single-writer pipe—correct for UX, a hot partition risk for viral groups.

Failure inside the box: If the commit succeeds but notify fails, the message is still safe in the inbox; reconnect + sync repairs the UI. If you ack before commit, you invent false “sent” ticks and trust collapses.

3) Online notify vs offline push

After commit:

1. Look up connection placement for the recipient (and each of their devices).
2. If a socket is live and healthy, enqueue ciphertext on that connection (respect queue caps).
3. If offline, enqueue a generic push (“New message”) and rely on SyncMessages later.

Do not put the decrypted body in the push payload under true E2E.

4) Numbered happy path (put it together)

1. Client encrypts → SendMessage(ciphertext, client_msg_id, conversation_id).
2. Chat service dedupes on client_msg_id.
3. Allocate msg_id + seq; commit to message store.
4. Ack sender (one gray tick / “sent”).
5. Socket push to online peers or generic push + inbox wait for offline.
6. Recipient decrypts; optional AckDelivered / AckRead according to product rules.

If you remember one thing: Success on SendMessage means inbox durable, not “friend’s lock screen already showed the text.”

Key challenges

These are the problems that separate weak diagrams from designs you can defend. For each row, say what the user sees if you get it wrong.

Challenge	What users see if it goes wrong	What to say in the interview
Encryption + many devices	"Waiting for message" on one phone while another works	Encrypt for each device; key repair and multi-device sync flow
Order on network retry	Duplicate bubbles or wrong order	Sort by seq, then msg_id; dedupe client_msg_id on retry
Big groups	Lag, failed sends, phone gets hot	State max size; shared log, announcement mode, or rate limits per group
Typing indicators	Sluggish app, battery drain	Throttle; send only when app is in foreground; respect hide last seen
Video on bad Wi‑Fi	Stuck on one checkmark while text works	Chunked upload with resume; separate timeout and path from text

Correct order and durable inbox matter more than which message broker brand you name—and encryption limits what receipts and search can mean on the server.

Scaling the system

Shard inbox storage by chat id (until one chat becomes "celebrity hot") or by user id (each user's inbox partition). Watch traffic per partition—one viral group can concentrate writes on a single shard.

Connection servers scale horizontally by user id or region. Use sticky routing so the same user tends to land on the same box when you need local connection state. Cap outbound queue size per socket so one slow phone does not exhaust server memory. Plan for thundering herd reconnects after deploys with exponential backoff.

Multiple regions may be required for data residency. Strict order across regions is hard—be honest that failover can produce brief inconsistencies; per-chat seq is still authoritative within a region.

Push workers scale separately from chat writes. Respect Apple and Google rate limits; delete dead device tokens so you do not retry forever.

Watch time from database save to first byte on the peer's socket—not only SendMessage HTTP success rate. A high lag there means users see "connecting…" while APIs look green.

Failure handling

Scenario	What the user sees	What to build
Connection drops mid-send	"Sending…" spinner on phone	Reconnect with backoff; resend same client_msg_id
Video upload stops halfway	Stuck on one gray tick	Resume upload; expire broken partial blobs
Push accepted but app closed	Message late until user opens app	Inbox sync on open—normal, not a bug
Whole region unavailable	Delay, possible brief order glitch	Fail over; admit cross-region limits honestly

When in doubt about whether a message was saved, fail closed on ambiguity for the sender—do not show "sent" until the inbox row commits. Users will tap send again; duplicate detection must make that safe.

A real outage is "cannot send at all" or messages disappear. Saving to inbox before ack to the sender prevents loss; slow delivery is a different class of problem.

API design

These are the main calls the mobile app uses (often gRPC or similar over TLS). Walk the hottest path first—SendMessage—before listing every RPC.

Messages

Endpoint / RPC	Role
SendMessage	Upload ciphertext + metadata; returns msg_id, seq
SyncMessages(cursor)	Paginated since cursor
AckDelivered / AckRead	Receipt pipeline

Query params (sync):

Param	Role
cursor	Opaque last seen position
limit	Max messages per response (capped server-side)
conversation_id	Thread scope

Media

Step	Role
POST /v1/media:prepare	Returns pre-signed upload URLs + media_id
PUT to object storage	Client uploads encrypted blob
SendMessage references media_id	Pointer only on signaling path

In simple terms: save the scrambled message to the database, then either push on the live socket or send “new message” to the phone.

Figure: Commit to inbox before ack; online peers get the socket path, offline peers get a generic push.

Diagram (send hot path):

Client --SendMessage(ciphertext)--> Chat svc --> DB commit
                                        |
                                        +--> if peer online: conn push
                                        +--> if offline: Push "new message"

Errors: 429 too many requests; 413 metadata too large; 409 duplicate client_msg_id → return the same msg_id as the first send so mobile retries stay safe.

The usual flow runs in three steps: SendMessage commits ciphertext to the inbox, online peers receive on the socket, offline peers get generic push and catch up with SyncMessages(cursor) when the app opens.

In the interview, walk SendMessage in order: encrypt on client → dedupe → save inbox → notify. Say aloud that 409 is intentional—"same send twice is OK."

Observability

Green SendMessage HTTP rates alone will lie. Measure the path users feel:

Signal	Why it matters
Inbox commit success / latency	Delivery promise to the sender
Save-to-socket p99	Time until an online peer can decrypt
Push accept vs in-app render lag	Separates vendor delivery from UI truth
Writes per partition / top conversation_id	Catches viral-group hot shards
Connection queue depth / disconnect reason	Memory pressure before CPU looks busy
Dedup hit rate on client_msg_id	Confirms retries are common and working

Alert when save-to-socket or partition write skew moves, not only when API 5xx jumps.

Security and abuse

Authenticate every send and sync; authorize membership before writing into a conversation. Under E2E, the server never needs plaintext for routing—treat accidental plaintext logging as a privacy incident. Rate-limit SendMessage, OTP, and group create; throttle typing and presence so bots cannot write-storm connection clusters. Abuse pipelines should use metadata, graph signals, and user reports—not “read the chat from the DB.” Multi-device key directories and stolen-session revoke paths matter as much as TLS on the wire. Media upload URLs must be short-lived and scoped so one leaked pre-signed URL cannot empty object storage.

Cost awareness

Every open socket costs memory; every copy-on-send group multiplies storage and write IOPS by member count; every media blob stored forever dominates disk more than text ever will. Choosing shared-log for huge groups trades write cost for more read traffic and hotter partitions. Generic push at billions of users is an invoice line with Apple and Google—delete dead tokens aggressively. Client-side search indexes cost device battery and storage so the server can stay ignorant of plaintext; that trade is intentional, not free. Archive and delete-for-everyone policies are cost features as much as product features.

Production scenes

Textbooks teach boxes and arrows. On-call teaches this: the graph can look fine while users are furious.

SendMessage returning success is not the same as the friend seeing the bubble. Apple marking a push "delivered" is not proof the human read the chat. Interviewers love this gap—it separates diagram-only answers from people who have thought about launch day.

Three nightmares that look "healthy" on a dashboard:

What the dashboard shows	What is actually happening
Chat API success rate and database latency look normal.	Users stare at "connecting…" for seconds because connection servers are saturated or reconnect storms after a deploy.
Push provider reports delivered to Apple or Google.	The user saw no banner—Do Not Disturb, battery saver, or stale token—and the message waited in the inbox until they opened the app.
Global QPS and CPU are flat on chat workers.	One viral group hammers a single database partition; sequence allocation and typing indicators make the hot shard worse.

Track the whole path—inbox commit → fan-out → socket or push → decrypted UI—not one green metric in the middle.

Scene 1: APIs green, chat feels slow

The moment: Evening peak. Users complain messages take forever to show one checkmark, then two. Status dashboards show healthy API latency.

The trap: Long-lived sockets fail differently than one-off HTTP. Teams alert on 5xx on REST but not on time from inbox save to first byte on the peer's connection. TLS handshakes, GC pauses, and reconnect herds after deploys hurt chat without moving the "API success" graph.

What to build:

• Measure save-to-socket latency end to end—not only SendMessage 200.
• Load-test connection pools before peak; rehearse deploy with reconnect backoff so clients do not amplify an outage.
• Remember: queue lag zero does not mean the user saw the message on screen.

Scene 2: Push delivered, user saw nothing

The moment: Support tickets say "I never got notified." Vendor dashboards show pushes delivered.

The trap: Push only tries to wake the app. Under E2E, the server often cannot put message text in the banner. "Delivered to Apple" is not "user read the thread" and not even "app decrypted and rendered."

What to build:

• Treat push as help, not proof of human attention.
• Drive unread state from synced inbox rows on the device.
• Measure push failures separately from in-app rendering bugs.

Scene 3: One viral group melts one shard

The moment: Live event channel. Messages fail or lag. One conversation_id drives hundreds of writes per second.

The trap: Sharding by chat id works until one chat becomes enormous. Per-chat order requires a single logical sequence pipe—correct for UX, a hot partition at scale. Typing indicators multiply writes.

What to build:

• Rate-limit messages per group; use announcement mode for broadcast-scale channels.
• Cap group size or move stadium traffic to a different product surface.
• Chart writes per partition, not only global message QPS.

Scene 4: CPU calm, memory spikes, random disconnects

The moment: Connection servers disconnect healthy-looking clients. Memory climbs while CPU looks bored.

The trap: Each socket has an outbound queue. A phone on slow Wi‑Fi or in the background reads slowly; frames pile up until the server must shed load.

Figure: A flat CPU graph can hide per-connection queues eating memory.

What to build:

• Cap queue depth per connection; merge repeated typing events.
• Disconnect gracefully with a resume cursor so clients do not replay full history blindly.
• Alert on queue depth and time from save to first byte sent.

In the interview: Pick one scene. Tell it like a short story—what users felt, what metric lied, what you would ship next. End with: you do not need global order—one rising seq per chat is enough, and say why.

Bottlenecks and tradeoffs

Encryption vs search and moderation

The tension: Users want private chats; the product also wants search, abuse detection, and support tooling.

What "fair" can mean for privacy:

• True E2E — server stores ciphertext only; search and moderation use metadata, reports, or client-side indexes.
• Metadata search — who talked to whom, when, message size—never the body.
• Product honesty — name what you give up on the server; do not pretend support can "just read the chat."

What to build: Client-side search index after decrypt, or limited server tools on metadata. Abuse pipelines on rate, graph, and user reports—not logging plaintext "for debugging."

In the interview: Say explicitly what the server cannot do if chat is E2E. Weak answers claim "we'll index messages for support."

Same order on every device vs speed and cost

The tension: Users expect the same thread on phone, tablet, and web; perfect instant sync everywhere costs extra round trips.

What breaks: Brief moments where two devices show slightly different order until SyncMessages catches up.

What to build: Many products accept good enough order that converges in seconds; stricter linearizability costs latency and engineering.

In the interview: Pick a bar—"good enough for chat" vs "strict same order everywhere"—and justify it with seq and sync cadence.

Small groups vs stadium groups

The tension: Copy-on-send feels simple for family chats; at stadium scale, writes explode.

What breaks: One viral group overloads one database partition.

What to build: State max group size; switch to shared log + cursor or announcement mode above that limit.

In the interview: Group size is not a footnote—it picks write amplification vs read amplification. Name your cap and model aloud.

Interview tips

You have the full design in mind now. These five back-and-forths are how interviewers test whether you can defend it under pressure. Each one starts with a tempting shortcut, an interview push, and a clear answer.

Two phones show different order — wall clocks are not the source of truth

You might say: "Messages are ordered by timestamp."

Interview Push: "Alice in London and Bob in Tokyo both send at 'the same second' on bad clocks—what order do users see?"

Land here: The server assigns a rising seq per chat, not per planet. If two messages tie, break ties with msg_id. Never sort by the phone's wall clock—clocks disagree, networks reorder packets, and a later retry can look “earlier” on a skewed device clock. When two devices briefly diverge after reconnect, SyncMessages(cursor) converges them onto the same seq order. Global total order across every chat on earth is not the goal; per-conversation order is.

The elevator retry — at-most-once loses messages users care about

You might say: "At-most-once delivery is simpler—we won't duplicate."

Interview Push: "User taps send in an elevator, TCP retries, they tap send again—do you drop one copy or show two bubbles?"

Land here: Mobile networks are at-least-once. Dropping the second attempt to “avoid duplicates” can lose the only successful commit when the first response never returned. Keep a stable client_msg_id for that tap; the server returns the same msg_id and seq when the same id arrives again. The UI shows one bubble; order stays tied to seq, not to how many times the phone retried. If you remember one picture from the dig, it is the elevator: response lost → same id → same seq.

Push carries the message — encryption says otherwise

You might say: "Push notification delivers the chat text."

Interview Push: "If messages are end-to-end encrypted, what appears on the lock screen?"

Land here: Usually only "New message"—not the body. The durable row lives in the inbox; the app fetches and decrypts after open (or over the live socket). Push is a wake-up call. Apple or Google accepting a notification is not proof the human saw the thread, and it is not how ciphertext becomes readable.

We log message text for support — that breaks E2E

You might say: "We log full message content so engineers can debug faster."

Interview Push: "What can the server legally and technically know if chat is truly E2E?"

Land here: The server moves ciphertext and metadata (sender, size, time)—not readable text. For abuse: rate limits, who-sent-to-whom, user reports. Do not claim support can open a thread from the database. If you need structured debugging, store reason codes on delivery failures—not chat bodies. Saying “we’ll add searchable plaintext later” quietly undoes the privacy promise you opened with.

Typing updates to everyone — write storms at billion-user scale

You might say: "We broadcast typing to all contacts in real time."

Interview Push: "A billion daily users open keyboards—what happens to your write load?"

Land here: Throttle typing and presence; send only when the app is foreground. Batch or sample updates; merge repeated “typing…” events on the connection server so memory does not grow forever. Respect hide last seen. Presence is optional polish—not on the same urgency path as SendMessage. If you put typing on the durable inbox path at full fidelity, you invent a second product’s worth of writes without improving chat reliability.

Red flags to avoid without correcting yourself: “Order by phone timestamp,” “push is the message,” “log plaintext for support,” “copy every send into fifty thousand inboxes,” “at-most-once so we never dedupe.”

After each interview push, close with one real part of the design you would build—seq, client_msg_id, inbox sync, ciphertext storage, or throttled presence—not "we'll scale it later."

What should stick

After reading this guide, you should be able to explain:

1. Two paths — Fast lane for text and control; truck lane for photos and video on object storage + CDN.
2. Inbox before push — Durable server inbox is the delivery promise; push only tries to wake the app.
3. Per-chat seq + client_msg_id — Server numbers beat phone clocks; duplicate IDs make retries safe.
4. Groups pick the model — Copy-on-send for small groups; shared log + cursor or caps for large ones.
5. Ops beyond HTTP 200 — Track socket delivery latency, hot partitions, and per-connection queue depth.

Tell it in the room: "I split text and media. One send: encrypt, dedupe with client_msg_id, save to inbox with seq, notify online peer on the socket or offline peer with generic push. Groups: I state max size and pick copy-on-send or shared log. E2E means ciphertext only on the server—push cannot quote the message."

Key Takeaways

• Clarify first — Group size, E2E depth, multi-device, inbox-as-promise, and out of scope.
• Two paths — Fast lane for text and control; truck lane for photos and video on object storage + CDN.
• Inbox before push — Durable server inbox is the delivery promise; push only tries to wake the app.
• Per-chat seq + client_msg_id — Server numbers beat phone clocks; duplicate IDs make retries safe.
• Groups pick the storage model — Copy-on-send for small groups; shared log + cursor for large ones.
• E2E limits the server — Ciphertext only; search and moderation use metadata, reports, or client indexes.
• Operate the path — Save-to-socket latency, hot partitions, queue depth—not only HTTP 200.

Related Topics

• Notification Service System Design - Push delivery, device tokens, and why “delivered to Apple” is not “user read”
• Facebook Feed System Design - Contrast read fan-out and ranking with per-thread chat order
• Distributed Cache System Design - Presence maps and hot key patterns for online-user routing
• Message Queues - Where async workers fit after the inbox commit (not always on every text send)
• WebSockets and Real-Time - Long-lived connections, heartbeats, and reconnect storms

Quick revision notes

Use this as a last look before a mock interview, not as a substitute for the full guide.

Clarify first. Agree maximum group size, how deep you go on E2E crypto, multi-device expectations, whether inbox (not push) is the delivery promise, and what you park out of scope. Skipping this often means you design a feed or a crypto whiteboard instead of chat.

Draw two lanes. Text, receipts, and presence stay on the fast signaling path. Photos and video upload separately; the chat message only carries a media_id pointer so a huge file never blocks checkmarks.

Protect one send. Encrypt on the client, reuse client_msg_id on retry, assign per-chat seq on the server, commit ciphertext to the inbox before you ack the sender, then notify online peers on the socket or wake offline peers with generic push.

Pick a group model out loud. Copy-on-send for small groups; shared log with cursors or announcement mode above your size cap—and chart writes per partition so one viral channel cannot melt a shard.

Name E2E limits. The server routes ciphertext and metadata; lock-screen text stays generic; search and moderation lean on client indexes, metadata, and reports—not plaintext logs “for debugging.”

Practice this path: WhatsApp System Design.