System design interview guide
Design WhatsApp
Picture sending the message "running late" to your family group chat from a train with a weak phone signal. The app does not hear back from the server in time, so it sends the same message again, which is what chat apps are built to do when a reply gets lost. The company that runs the chat would like everyone in the family to see one bubble, in the right order, with a small tick showing the message is saved, and it would like to do this without being able to read the text. The part of the system that carries, stores and delivers those messages is called a chat messaging system. The painful failure is that the line shows up twice because the server saved the retry as a brand-new message.

Problem statement
You're designing a chat messaging platform in the style of WhatsApp: the servers and phone apps that carry a message from one person to another or to a group, show a tick when it is saved and another when it is read, and use end-to-end encryption so the server routes scrambled bytes, not readable message text. Functionally you need one-on-one and group messaging, text and media, receipts, presence, and multi-device support. Non-functionally you need very low latency for short signals when both people are online, high availability, and strong privacy. Scale sits on the order of billions of users and hundreds of thousands of messages per second at the busy hour, and what dominates at that scale is open live connections, per-chat writes, and group fan-out, not one monolithic "send message" call. What interviewers reward: a separate fast path and media path; the message saved in the inbox before the sender is told; a per-chat sequence number and a sender-made duplicate id so retries are safe; an honest group model with a stated maximum size; and a clear account of what encryption stops the server from doing (search, readable push text), not a generic "we'll add Kafka and scale."
Introduction
The problem: one message from a train
Picture sending “running late” to your family group chat from a train with a weak phone signal. You tap send, and the app shows a small clock because nothing has come back from the server yet. After a few seconds the app does what chat apps are built to do on a shaky network: it sends the same message again. In the meantime the first copy had actually arrived, and only the reply was lost on the way back.
Now the people running the chat have several problems at once. If they save the second copy as a new message, your family sees “running late” twice. If two people in the group write at nearly the same moment, everyone should still see the same order, even though their phone clocks disagree by a few seconds. If your sister’s phone is switched off, she should still find the message waiting when she turns it on. And the company would like to do all of this without being able to read what you wrote.
The thing that handles all of this is called a chat messaging system: the phone apps plus the servers that carry each message, store it until the other side has it, and report back with ticks. This page walks through how to think about it in the order you would actually reason in an interview: first why anyone builds it and what exactly we are solving, then what it must do and how big it is, and only after that the design itself.
Why teams bother with a messaging system
You probably use a chat app many times a day, and you probably never think about how much has to go right each time. The reasons companies invest so heavily in getting it right are very practical.
Trust in a daily habit. People use chat for family news, for work, and sometimes for emergencies. A message that is lost, shown twice or shown in the wrong order breaks that trust quickly, and people move to another app.
Privacy. Private conversations are some of the most sensitive data a person has. Promising that the company cannot read them is a major reason people choose one app over another, and it changes what the servers are allowed to know.
Bad networks are normal. Many people send messages from trains, lifts, crowded stadiums and places with a weak signal. A design that only works on a perfect connection is a design that fails for a large share of real users.
Cost at huge scale. With a billion people sending messages, every open connection uses a little memory on a server, every photo uses storage, and every extra database write is multiplied by billions. A design that is merely “correct” can still be too expensive to run.
What problem are we solving?
Put simply, a person writes a message to one other person or to a group, and the system has to make sure every recipient eventually sees exactly one copy, in the same order the chat was written, even if their phone was offline when it was sent.
The sender should see a small sign that the message is safely stored, and later a sign that it was read. A photo or a video should travel along with the chat without slowing it down. And the company’s servers should only ever handle scrambled bytes, never the readable text.
That is what success looks like. The hard parts are the places where the real world gets in the way. Phones retry because replies get lost. Clocks on different phones disagree. Recipients go offline for hours. Groups can have a dozen people or fifty thousand. Photos are hundreds of times bigger than a line of text. And privacy means the server cannot simply look inside a message to help. We will not guess at solutions yet. First we need to be clear about what the system must do, and how well, so that every design choice later has a reason behind it.
This is also different from designing a news feed. A feed is mostly about choosing what to show you and tolerates a few seconds of delay. A chat is about delivering every message once, in order, quickly and privately.
This page is the interview deep dive in the system design interview guides hub. If long-lived connections are completely new to you, skim WebSockets and real-time first.
How to approach
In an interview you do not start by drawing boxes. You start by turning a short, vague prompt such as “design WhatsApp” into a clear contract: how big the groups are, what promise we make when a phone is offline, how deep to go on privacy, and what is out of scope. Only then do you size the problem, make the design decisions one at a time, and draw the result. The list below is the order this page follows.
How to design a chat system in an interview
- Clarify the problem: how large a group can be, what “delivered” promises when a phone is offline, how much to say about encryption, whether people use several devices, and how big the system is.
- Write the functional requirements (what the system does) and the non-functional requirements (how fast, how reliable, how private, how big).
- Estimate the size: messages per second, open connections, storage per day, and how much bigger media is than text.
- Make the design decisions in order: what “sent” means, how retries stay safe, how a chat is ordered, how the other phone finds out, where photos travel, how privacy changes the server’s job, how groups fan out, and how receipts scale.
- Draw the picture those decisions produce, then walk one text message through it.
- Go deeper where the interviewer pushes: the retry race, order numbers, offline sync, encryption limits, large groups, and one production failure.
Clarifying questions (say these in the room)
In a real interview the prompt is only a few words. Before you design anything, ask questions until the problem is clear. Each answer below changes the design, which is why it is worth asking.
You: “What is the maximum group size: a dozen family members, a few hundred teammates, or a stadium-sized channel? That decides whether we copy each message into every member’s mailbox or store it once and let members read it from one shared history.”
Interviewer: “Assume up to a few hundred for normal groups, and call out what changes if someone wants fifty thousand.”
That answer tells you that copying a message into each member’s mailbox is fine for ordinary groups, and that you need to name a different model above your size cap. You do not need to design the stadium case yet. Write one line on the board such as “small group: copy per member; huge group: different model.”
You: “When a phone is offline, is the promise ‘the message is saved on the server and the phone catches up later,’ or is it ‘a push notification must reach the phone’?”
Interviewer: “Saved on the server is the promise. Push is best effort.”
This is the most important answer for the whole design. It means a successful send means “safely stored,” not “the other person has seen a banner.” Apple and Google decide whether a notification actually appears, and the system cannot control that.
You: “How deep should I go on encryption: a high-level ‘the server cannot read the text,’ or a full walk through the key-exchange protocol?”
Interviewer: “High level is enough unless I push.”
Now you know to spend minutes on delivery, order and groups, and not twenty minutes on cryptographic math. You still say out loud that the server stores scrambled bytes, because that one fact changes what the server can do for notifications and search.
You: “Do people use more than one device for the same account, and should a read tick in a large group show who read it or only how many?”
Interviewer: “Yes to several devices. Pick a product rule for large-group receipts.”
Several devices means the message has to be prepared for each device and each device needs its own place in the chat history. Large-group receipts need a rule, because one read event per member per message can cost more than the message itself.
You: “Roughly how many people and messages are we talking about, and does order only need to be correct inside one chat, or the same across the whole app?”
Interviewer: “About a billion active people and tens of billions of messages a day. Order only needs to be right within one chat.”
Those numbers feed the capacity estimate in the next section. The second part is a relief: strict order across every chat on earth would be very expensive, and no user can tell the difference. Order inside one conversation is what people see.
You: “Do we keep messages on the server forever, or only until every device has received them?”
Interviewer: “Keep enough history for a phone to catch up, and say what you would trim.”
This is a product choice with a cost attached. Some end-to-end encrypted messengers keep a message on the server only until all devices have it. This guide keeps a catch-up window, and you can say you would shorten it to save storage.
You: “I want to leave the full cryptographic protocol, building the SMS code network used for signup, and a payments or stories product out of scope so we can focus on delivery, order, offline catch-up, groups and the limits that privacy puts on the server. Does that match?”
Interviewer: “Yes, focus on messaging reliability and privacy limits.”
If the interviewer later asks for key-exchange proofs or the exact wire protocol, you do not need to answer yet. Park those until the design decisions, where each one has a natural place.
Functional requirements
Functional requirements describe what the system does. From the answers above, a chat system must:
- Send text in one-to-one chats and in groups. A message written by one person reaches every other member of that chat, once.
- Send photos, videos and files. These are much bigger than text, and the chat should show them without waiting for the whole file on every device.
- Show delivery signs. The sender sees that the message is saved, later that it reached the other phone, and later that the other person read it.
- Catch up after being offline. A phone that was off for a day finds every missed message waiting, in order, when it comes back.
- Work on more than one device. A second phone or a computer shows the same chat history.
- Show who is online and who is typing, in a light way that does not get in the way of real messages.
Non-functional requirements
Non-functional requirements describe how well the system must do its job. They are the promises that shape the design.
- Speed. When both people are online, the short signal (the text, a tick) should arrive in under about 100 ms, so the chat feels instant.
- No loss and no duplicates. A message the sender was told is saved must not disappear, and a retry on a bad network must not create a second bubble.
- Order inside one chat. Everyone in a conversation sees its messages in the same order. Order across different chats does not matter.
- Privacy. The server stores and moves scrambled bytes. It cannot read the text of a message, and notifications cannot quote it.
- Availability. People treat chat as a daily utility, so the service should keep working when single machines fail.
- Scale. About a billion active people, hundreds of millions of open connections at once, and tens of billions of messages a day.
- Media does not block text. A large photo or video must never delay a tick on a short message.
Out of scope (and why you park these)
Write a short out-of-scope list on the board so the interviewer sees you can bound the problem. Each line below says what we leave out and why.
Full cryptographic protocol design (key-exchange proofs, ratchet math). You need the system consequences of encryption, which are scrambled storage, generic notifications and limited server search. A lecture on the math is a different interview unless they ask for it.
Building the SMS or voice-call code network used to verify phone numbers. Signup matters, but verifying a number is a separate identity path. Mention that you would limit how many codes can be requested, then return to delivery.
A payments, status-stories or broadcast-channel product. Adjacent products take whiteboard time away from the hard path, which is durable delivery under retries and group fan-out.
Perfect global order across every chat on earth. Chat needs order inside one conversation. Strict order across all conversations and regions fights physics and buys users nothing. Say what you guarantee per chat, and what a regional failover may briefly relax.
That is the requirements stage. If you want to practice it before reading on, try the requirements stage on Practice System Design (about 5–10 minutes), then come back for the sizing and the decisions.
Minute pacing (about 45–60 minutes)
| Minutes | Focus |
|---|---|
| 0–5 | Clarify the problem; write functional and non-functional requirements; agree out of scope |
| 5–12 | Capacity, then the design decisions and the high-level boxes with one message story |
| 12–30 | The send in depth: saving, retries, order, then notifying online and offline phones |
| 30–40 | Groups, hot chats, receipts, privacy limits |
| 40–50 | One production story and failure modes |
| 50–60 | Questions and one tradeoff in more depth |
In the room (how you open): “I’d start by agreeing on a few things: how large a group can be, what we promise when a phone is offline, how deep to go on encryption, and whether people use more than one device. I’ll write the functional and non-functional requirements on the board and what we leave out, size the problem, and then make the design decisions one at a time, so every box I draw has a reason.”
If you remember one thing: Ask first, write the requirements down, and only then design. Tell one message story before you name any technology.
Capacity estimation
Once the requirements are clear, the next step is to find out how big the problem is, because the size decides which parts can be simple and which cannot. You do not need perfect statistics. You need enough sense of scale that you do not promise something impossible, such as copying one “hello” into fifty thousand mailboxes in a single instant. The numbers below are estimates to anchor the whiteboard, so say they are estimates and tune them in the room.
| Input | Order of magnitude (estimate) | Design implication |
|---|---|---|
| Messages a day | Tens of billions (for example 50 billion) | 50 billion ÷ 86,400 seconds is about 580K sends a second on average, and the busy hour is higher. Split the data across many machines |
| Peak message rate | Hundreds of thousands per second, possibly near a million | Keep each send short: one save plus a notification |
| Open connections | 100 million or more at the same time | Regional connection servers; at an illustrative 200K idle connections per machine, 100 million connections is about 500 machines |
| Size of one stored text message | Roughly 300 bytes with its small header (estimate) | 50 billion × 300 bytes is about 15 TB a day before copies and replicas |
| Media | Hundreds to thousands of times larger than text | Object storage and a content network; the chat only carries a pointer |
| One group message | 1 write for a shared history, or one per member if copied | A 256-member group copies 256 times, a 50,000-member group 50,000 times |
| One viral chat | Hundreds of messages a second in one conversation | One conversation’s order is handled by one owner, so that machine can run hot |
What the numbers imply
- Spread the data by chat or by recipient. No single database handles hundreds of thousands of writes a second, so messages are split across many machines, each owning a slice of chats or mailboxes.
- Open connections are a first-class cost. Most of those 100 million connections are idle, waiting for something to arrive, so servers are tuned to hold very many quiet connections, not only to use their CPU well.
- Media must stay off the fast road. A photo is far larger than a message, so putting both on one path would let a single upload slow thousands of ticks.
- Group size is a capacity decision, not a footnote. Writing one row per member is fine at 12 and unsafe at 50,000.
- Be honest about the ~100 ms budget. It holds when both people are online and in the same region. Across the world, the speed of light alone uses a large part of it, so say that the budget is for the common case.
Checkpoint: Why can’t a single database take the message writes, and why must photos stay off the text path?
If you remember one thing: The estimate tells you to split the data, to treat connections as a cost, and to keep large files off the urgent path.
Design decisions
We now know what the system must do, how reliable and private it must be, and roughly how big it is. Now we choose. Each decision below answers one question the requirements raised, and each one rules out an easier idea that would not survive the requirements. Read them in order, because every decision builds on the one before it.
Decision 1: What does “sent” actually promise?
The requirements said a message the sender was told about must never be lost, and that a phone can be offline for hours. So the first thing to decide is what the little tick promises. The simplest idea is to hand the message straight to the other phone’s live connection and show the tick as soon as the server has accepted the request. It is fast and needs no storage.
It breaks on exactly the cases the requirements care about. If the other phone is offline, there is nowhere to hand the message to, so it would be lost. If the server crashes after showing the tick but before the other phone received anything, the sender believes a message is safe that no longer exists. That breaks the “no loss” requirement.
There are three ways out. We could put every message in a general-purpose queue and let consumers pull from it. That keeps the message, but a queue for each of a billion users and per-chat order are awkward to maintain, and an extra hop sits on every send. We could rely on push notifications to carry the message, but Apple and Google do not promise delivery and under privacy rules they cannot carry the text. The third way is to give each recipient a mailbox on our own servers, called an inbox, and write the message there before telling the sender anything.
We choose the inbox. The rule is save first, then tell the sender, then try to tell the other phone. A tick on the sender’s screen then means “this message is safely stored,” which is a promise we can keep. Whether the other phone is online right now only decides how quickly the message reaches it, not whether it ever will.
Decision 2: How do we stop a retry from creating a second bubble?
Phones retry. The reply to a send can be lost even though the server saved the message, and a phone that hears nothing cannot tell “never arrived” from “arrived but the answer was lost.” The simplest idea is to let the server treat every request as a new message. That breaks the “no duplicates” requirement, because the family sees “running late” twice.
The next simplest idea is for the phone to never retry, so duplicates are impossible. That fails the other half of the requirement: if the first request really was lost, the message is gone and the user has to tap send again by hand. Delivery that may lose a message is called at most once, and for chat it is the wrong trade.
The choice is to let the phone retry freely and make retries harmless. When you tap send, the phone invents a unique id for that one tap and calls it client_msg_id. Every retry of the same tap carries the same id. The server remembers which ids it has already saved in that chat, and when a known id arrives again it answers “I already have this one” and returns the original result instead of saving again. Delivery that may repeat, combined with a server that ignores repeats, is called at least once with deduplication, and it is what every serious chat system uses. The Deep dive shows exactly how the server makes that check safe when two retries arrive at the same moment.
Decision 3: How do we order the messages in a chat?
Every member of a chat must see its messages in the same order, but phones do not agree on the time. The simplest idea is to sort by the timestamp the phone attaches. A phone with a clock that is a few minutes wrong would make its messages jump ahead of or behind everyone else’s, and a retry could look older than the message it follows. That breaks the order requirement.
Using the server’s clock is better but not enough. Two servers can have slightly different clocks, and two messages that arrive in the same millisecond still need a clear winner. A single counter for the whole app would be perfectly ordered but could not keep up with hundreds of thousands of sends a second, and nobody needs order across chats anyway.
The choice is a sequence number per conversation, called seq. The server hands out 1, 2, 3 and so on inside each chat, one at a time, and every phone sorts by seq, using the message’s own id only to break an impossible tie. Order then depends on one counter per chat, which is cheap, instead of on clocks, which cannot be trusted. The cost is that a very busy chat has a single owner handing out numbers, and the Deep dive returns to that.
Decision 4: How does the other phone find out about a new message?
The recipient should see a new message in under about 100 ms when online, and still get it later when offline. The simplest idea is for each phone to ask the server every few seconds whether there is anything new. With hundreds of millions of phones, that means an enormous number of questions whose answer is mostly “no,” and a typical delay of several seconds, so it fails both the speed and the cost goals.
Push notifications alone are also not enough, because they are a best-effort wake-up call that the phone can ignore, and they cannot carry the text.
The choice is to use three tools for three situations, with the inbox as the truth. While the app is open, the phone keeps a long-lived connection to a connection server, a connection that stays open so the server can send something the moment it exists. If the phone is not connected, the server sends a generic push notification such as “New message” to wake it. Whatever happens to the live connection or the push, the phone can always call SyncMessages(cursor), where a cursor is a bookmark for the last message the phone has seen, and the server returns everything stored after that bookmark. The connection and the push are only hints, and the inbox plus the cursor repair every missed hint.
Decision 5: Where do photos and videos travel?
Photos and videos are hundreds or thousands of times larger than a message, and the non-functional requirements said a big file must never delay a tick. The simplest idea is to send the file through the same request as the text. One large upload would then occupy the same connection, the same server memory and the same time budget as thousands of small messages, so every tick behind it would wait.
The choice is two roads. The fast lane carries short items: text, receipts and presence. The media lane carries the large file: the phone uploads it directly to object storage, a service built to store large files cheaply, using a temporary upload link that the server hands out (called a pre-signed URL), and receivers download it through a content delivery network, a set of servers spread around the world that keep copies close to people. Once the file is stored, the fast lane carries only a small pointer called media_id. A broken upload can resume where it stopped, and it can fail without touching a single text message.
Decision 6: What can the server see, and what changes because of that?
The requirements said the company cannot read the text. The simplest way to protect messages is to encrypt the connection between phone and server, which is what the lock icon in a browser means. But then the server receives the plain text and could read it or hand it over. Encrypting files on the server’s disk has the same weakness, since the company holds the keys.
The choice is end-to-end encryption: the sender’s phone scrambles the message with keys that only the intended phones hold, and only those phones can unscramble it. The server receives, stores and forwards ciphertext, which is the scrambled form, plus the small facts it needs to route it, such as who sent it, which chat it belongs to, when, and how large it is. At the level of this interview, you say that phones agree on keys, messages are encrypted on the phone, and the server only moves ciphertext.
That choice has consequences you must say out loud, because they change other parts of the design. Push notifications cannot quote the text, since the server does not know it. The server cannot search message bodies, so search runs on the phone after decrypting, or covers only the small routing facts. A person with several devices needs the message prepared separately for each device. And moderation has to rely on reports, rate limits and who-talks-to-whom signals instead of reading chats.
Decision 7: How does a message reach every member of a group?
A one-to-one message goes to one inbox. In a group the same message must reach many members, and the earlier clarifying answer said groups can reach a few hundred members, with something different above that. The simplest idea is to copy the message into each member’s inbox at send time. It is easy to reason about, because every member then has an ordinary inbox to read from, and it works well for a family of twelve.
It breaks as groups grow. A message to 50,000 members would mean 50,000 database writes for a single “hello,” and a viral group could flood one machine. The alternative is a shared group log: store the message once, and give each member a cursor that says how far they have read. Sending becomes one write, and reading becomes more work, because every member fetches from the same log, which can become a hot spot of its own. A third option is announcement mode, where only a few administrators may write, so large audiences read much more than they write.
The choice is to state a maximum group size and use copy on send below it, and a shared log with cursors above it, with announcement mode and per-group send limits for the largest audiences. Under encryption, each member’s copy is still prepared so only their devices can read it, which is one more reason large groups are costly. The Core design approaches section compares the three side by side.
Decision 8: How do ticks and “online” indicators scale?
Delivery signs and presence are part of the product, but they are not worth threatening the core path. The simplest idea is to save every event, every delivered tick, every read tick and every “typing…” signal, as a durable record. In a group of 500, one message could trigger 500 read events, which can cost more than the message itself, and a billion people opening keyboards would create a storm of writes for something that is useless a few seconds later.
The choice is to treat different signals differently. A delivered tick is a small state change on a message the server already holds. A read tick is reported by the recipient’s app, and the server forwards it according to a product rule. In small groups every member’s read is shown. In large groups the product throttles, batches or shows a count such as “read by 42,” or turns detailed receipts off. Typing and online status are not durable at all: they are short-lived signals on the live connection, sent only when the app is in the foreground, merged when repeated, and allowed to be lost.
Putting the decisions together
Taken together, the eight decisions are the design. A message is encrypted on the sender’s phone and sent with a unique id for that tap. The server drops repeats of that id, assigns the next number for the chat, and saves the scrambled message into the recipients’ inboxes before telling the sender it is stored. It then wakes online phones over their live connection and offline phones with a generic push, and any phone can catch up from its bookmark. Photos and videos go by a separate road and appear in the chat as a small pointer. Large groups use a shared history instead of copies, and ticks and typing signals are throttled.
In the room (a fuller opening you can actually say): “I’d keep two lanes: a fast lane for text, receipts and presence, and a separate upload road for photos and video that only leaves a pointer in the chat. For one text message the phone encrypts on the device and attaches a unique id for that tap. The server drops a repeat of that id, assigns the next sequence number for the chat, and saves the scrambled message to the recipient’s inbox before it tells the sender anything, so the tick means saved. Online phones get it on their live connection, offline phones get a generic push, and any phone can catch up from its last cursor, so the inbox is the truth and push is only a hint. For groups I would state a maximum size, copy per member below it and use a shared log with cursors above it. Encryption means the server cannot read or search the text, so search runs on the phone, and large-group read receipts are throttled or shown as a count.”
The next section draws these decisions as one picture and shows who owns what.
High-level architecture
The decisions above were made one at a time. This section puts them on one page as a picture: which boxes exist, who owns what, and how one message travels through them. The deeper mechanics come later, so keep this at the level of a map.
What breaks if you merge everything
The decisions above already said photos must stay off the text path and the server must not see readable chat. Put that another way: one service that accepts full video uploads and “who read this” ticks on the same urgent road will choke at peak. Handshakes, database writes and push notifications all fight for the same few milliseconds. If you also log readable messages for “debugging,” you fail the privacy requirement before scale even matters. That is why the picture below has separate lanes and no plaintext.
What works: two lanes
Picture two roads:
- Fast lane: short messages: text, who read what, “user is online,” connection events.
- Truck lane: photos and video: upload to file storage, download through a content network.
The server moves scrambled bytes. It does not read the chat content.
How it fits together:
Phones keep a long-lived connection—an open pipe that stays up so the server can push something the moment it exists (often WebSocket or MQTT, wrapped in TLS encryption on the wire)—to connection servers in each region. The chat service receives encrypted text plus small routing facts, assigns a stable message id (msg_id) and a sequence number per chat (seq), and saves to a database split across many machines (by chat id or by each user’s inbox). If the friend is online, send over the live connection. If offline, save to their inbox first, then send a push notification through Apple or Google (APNs or FCM). For media, the phone uploads with a short-lived upload link the server handed out (a pre-signed URL); the fast lane only carries a file id.
Who does what:
- Connection servers: Keep connections alive, check they are healthy with heartbeats, and limit queue size per user so one slow phone does not use all the memory.
- Chat / inbox service: Owns message order (
seq), saves messages reliably, and tracks sent / delivered / read. - Object store + content network: Stores files and supports resume if an upload breaks.
- Push service: Maps each user to phone tokens and respects notification settings.
- Keys (sketch): Stores public keys per device. Never log readable messages.
Simple version: text uses the fast lane; photos and video use the truck lane.
[ Mobile client ]
|
| 1) TLS + long-lived conn (signaling)
v
[ Edge / Connection cluster ] ──► [ Chat / Inbox service ]
| |
| | persist ciphertext + metadata
| v
| [ Message store shards ]
| |
| 2) notify online peers +--> [ Push ] --> APNs/FCM (offline)
|
[ Same client or peer ] ◄── ack / delivery / read events
Media path (async, large):
Client --presigned PUT--> [ Object storage ] --> CDN URL in message body (encrypted)

Figure: Fast lane for text (blue), delivery to online users (green), and separate upload path for media (orange). The server only stores scrambled message bytes.
In the interview, three short phrases help you sound precise. Each one answers a follow-up interviewers often ask:
- Inbox before ack. When
SendMessagesucceeds, you are promising the ciphertext row exists in durable inbox storage, so the sender can show one gray tick. Do not describe delivery as “done” only because a push worker queued a job or because Apple accepted a notification. - Push is a hint, not the message. Under end-to-end encryption, the notification often says only “New message.” The phone must open the app (or use a live connection) to fetch and decrypt. Push failure does not mean the message was lost if the inbox row exists.
- Fast lane vs truck lane. Text, receipts and presence use the low-latency signaling path. Photos and video upload separately; the chat message body carries a
media_idpointer. Never block checkmarks on a multi-gigabyte upload.
Core design approaches
The picture shows the shape of the system, but two of the decisions have a few variants that are worth comparing side by side. This section recaps the one-to-one path in a few lines, then compares the group models from Decision 7 and the receipt rules from Decision 8.
1:1 messaging
Walk one ordinary one-to-one send the way Decision 1 through Decision 4 described it. Alice writes one line to Bob. Her phone scrambles the body before it leaves the device, invents a unique id for that one tap (client_msg_id) so a later retry is still the same send, and calls SendMessage. The server checks whether it has already seen that id; if so it returns the earlier result, and if not it assigns the next sequence number for the chat (seq), saves the scrambled message to Bob’s inbox, then either sends on Bob’s open live connection or sends a simple push that only says “new message.” Bob’s phone unscrambles the body and reports delivered or read when the screen allows.
Everyone in that conversation sorts by the server’s seq for the chat. They do not sort by a single clock for the whole app, because phone clocks disagree.
Group messaging
A group of twelve can copy each message into twelve inboxes and still feel fine. A stadium with fifty thousand people cannot, because one “hello” would mean fifty thousand writes unless you change the model.
These models trade write cost against read cost. Say your max group size and which you pick above it:
| Model | Plain idea | Good when | Watch out |
|---|---|---|---|
| Copy on send | One database row per member inbox per message | Small groups (family, team chat) | Write cost grows with member count |
| Shared group log | Store once; members fetch with cursor | Large groups, broadcast channels | Read cost; hot partition on viral group |
| Announcement mode | Only admins post; members mostly read | Stadium-scale updates | Different product semantics than free chat |

Figure: Group size is not a footnote. It picks your storage model and your hot-partition risk.
Receipts
Decision 8 already split delivery signs from typing noise; here is the product meaning of the ticks. Delivered (two gray ticks) usually means the server placed the message on the recipient’s live connection or in their inbox. Read (two blue ticks) means the recipient’s app reported they opened the thread. The server forwards that signal but does not independently know the user read it with their eyes.
In a twelve-person family group, one read event per member is fine. In a five-hundred-person group, one read event per member per message can cost more than the message itself, so you slow how often phones report (throttle), combine several reports into one write (batch), show a count such as “read by 42,” or turn detailed receipts off. Say which product bar you pick in the interview.
Data model
The design needs somewhere to keep these messages, so the next question is what exactly gets stored. Saying “a database” is not yet a design, because you have not said what the rows look like or which field decides where they live. Think in three stores: the message and inbox plane (the hot path), the media plane (pointers only on the fast path), and light presence and device state that must stay cheap.
Messages and inboxes
| Field / concept | Why it exists |
|---|---|
conversation_id | Which chat this row belongs to; also decides which machine owns the chat’s order |
msg_id | Server-assigned stable identity returned to the client |
seq | Rising integer per conversation, the order users should render |
client_msg_id | Sender-generated id so retries map to one bubble |
sender_id, recipient / membership | Routing and access checks |
ciphertext + small metadata | Body the server cannot read under E2E; size, media pointer, timestamps |
delivery_state | Sent / delivered / read signals the product surfaces as ticks |
For copy-on-send groups, you materialize (or fan out) rows toward each member inbox. For a shared group log, you store one sequence of messages for the group and keep a cursor / last_seq per member so each phone knows what to fetch next.
Devices, keys, and push
| Concept | Where it lives | Note |
|---|---|---|
| Device public keys | Key directory / identity service | Used so senders can encrypt for each device |
| Push tokens (APNs/FCM) | Push mapping store | Delete dead tokens; never place plaintext bodies in the map |
| Media blobs | Object storage + CDN | Encrypted client-side when product requires it; chat row holds media_id |
Interview land: On the board, sketch one conversation id pointing at rows that each hold a sequence number, a server message id, the phone’s tap id, and the scrambled body. Then say which extra lookup you need so a phone can ask “give me everything after my last cursor,” rather than drawing a giant “messages” table with no story about how rows are found.
Detailed design
We now know what is stored and where, so we can put the pieces in motion. This section follows one ordinary text message from Alice’s phone to Bob’s phone, and then shows the same thing from the app’s side: the calls the phone makes, what comes back and what the error codes mean. Read it once as a story. The retry race, the order numbers and the group questions come in the deep dive right after it.
The send path, step by step
When Alice taps Send, her phone encrypts the message body before it leaves the device. The server receives ciphertext, which means scrambled bytes it can route and store but not read as chat text. Alice’s phone also attaches a client_msg_id, a unique id for that one tap. This id matters because mobile networks often lose responses. If the first request succeeds on the server but the phone never hears back, the phone can retry with the same client_msg_id and the server can answer, “I already saved this exact send,” instead of creating a second bubble.
The chat service first checks whether (conversation_id, client_msg_id) already exists. If it does, the service returns the same msg_id and seq it created the first time. If it does not, the service creates a stable msg_id for the message and assigns the next seq, a rising number inside that one conversation. The seq is the order the screen should render. It is deliberately not the sender’s phone clock, because clocks can be wrong and packets can arrive in a different order from the one the user expects.
After identity and ordering are settled, the service saves the ciphertext and small routing metadata to inbox storage. That save is the delivery promise. If SendMessage returns success before the inbox row is durable, Alice may see a sent tick for a message the system can still lose. Once the row commits, the server can acknowledge Alice and move into notification work: online devices receive a frame on their live connection, while offline devices receive a generic push notification that only says something like “New message.” Under true end-to-end encryption, the push provider and the server should not carry the readable body.
Bob’s app has two ways to learn about the row. If the app has an open connection, the connection server can deliver the new ciphertext immediately. If the phone is asleep, out of network, or the push never appears because of battery settings or notification permissions, Bob still catches up later by calling SyncMessages(cursor). The cursor is Bob’s last known position in the conversation or inbox. The server returns ciphertext rows after that cursor, Bob’s phone decrypts them locally, and the screen renders them in seq order.
Receipts follow the same “save the truth, notify as a hint” pattern. A delivered receipt usually means the server placed the message on a device connection or made it available in the inbox. A read receipt is different: Bob’s app reports that the thread was opened, and the server forwards that signal according to product rules. In large groups, you should throttle or aggregate those receipts because one read event per person per message can become more expensive than the message itself.
Media uses a separate path. Alice uploads an encrypted photo or video to object storage through a resumable upload, often using a pre-signed URL. Only after the upload has a stable media_id does the chat message reference that media pointer on the fast path. A huge video should not hold the same transaction that assigns a text message seq, and a broken media upload should not block every receipt in the conversation.
- The phone encrypts the message and calls
SendMessage(ciphertext, client_msg_id, conversation_id). - The chat service looks for that
client_msg_idin this chat and, if it finds one, returns the earlier result. - Otherwise it assigns
msg_idandseqand commits the row to the message store. - It tells the sender the message is saved (one gray tick).
- It sends the message on the recipient’s live connection, or sends a generic push and lets the phone catch up later.
- The recipient decrypts, and its app reports delivered or read according to the product rules.
In the room (spoken recap): “One send works like this: the client encrypts, reuses a stable client_msg_id on retry, the server dedupes that id, assigns one per-chat seq, saves ciphertext to the inbox before acknowledging, then notifies online devices on live connections or offline devices with generic push. The inbox is the truth; push and live connections are delivery hints. Bob catches up with SyncMessages(cursor) and renders by server seq, not phone timestamp.”
API design: what the phone calls and what comes back
A chat system has no public API in the usual sense. The calls below are the ones the phone app makes (often gRPC or similar, over TLS), plus a few that operators and group admins use. Walk the hottest call, SendMessage, first, and only then list the rest.
The internal call. The phone sends one request to send a message and receives the facts it needs to show the right tick.
SendMessage(conversation_id, client_msg_id, ciphertext, media_id?)
→ { msg_id, seq, status: "saved" }
The calls around it.
| Call | Role |
|---|---|
SendMessage | Upload ciphertext + metadata; returns msg_id, seq |
SyncMessages(cursor, limit, conversation_id) | Everything stored after the cursor, a page at a time |
AckDelivered / AckRead | Receipt pipeline |
POST /v1/media:prepare | Returns pre-signed upload URLs + media_id |
PUT to object storage | Phone uploads the encrypted file directly |
For SyncMessages, cursor is an opaque last seen position, limit is the maximum number of messages per response (capped by the server so one phone cannot ask for a million rows), and conversation_id says which thread to read. For media, the upload goes straight to object storage, and SendMessage only carries the media_id pointer on the signaling path.
The short version of the hot path: save the scrambled message to the database, then either push on the live connection or send “new message” to the phone.

Figure: Commit to inbox before ack; online peers get the live-connection path, offline peers get a generic push.
Client --SendMessage(ciphertext)--> Chat svc --> DB commit
|
+--> if peer online: conn push
+--> if offline: Push "new message"
What the phone receives and the codes that matter. A repeated SendMessage with a client_msg_id that was already saved should return the same msg_id and seq as the first attempt, as an ordinary success. Some teams return 409 (“conflict”) with the same body to make the repeat visible, but a plain success is friendlier, because a retrying phone then needs no special case. Say which you pick, and that the repeat is intentionally safe. A 429 means the sender is going too fast, and a 413 means the metadata or message is too large. A 401 or 403 means the sender is not signed in or is not a member of that chat, and both are checked before anything is written.
What group admins and operators change. A small admin surface lets a group admin set the group’s mode (for example announcement mode) and lets operators revoke a stolen device or lower the send limit on one abusive group, without releasing a new app.
PUT /v1/groups/{id}/settings # announcement mode, member cap
POST /v1/devices/{id}:revoke # stolen or lost device
In the interview, walk SendMessage in order: encrypt on the phone → dedupe → save inbox → notify. Say aloud that a repeated send is safe on purpose: “the same send twice is fine.”
Deep dive: the questions an interviewer asks next
You have drawn the design and walked one message through it. This is the point where a good interviewer starts digging, and the questions usually come in a fairly predictable order, because each answer raises the next one. Read this section as that conversation. Each question is followed by an answer you can give, plus a hint of where the interviewer is likely to go next. When the hint is clear, you can offer the next piece before they ask.
“You said a retry gives the same message back. What does that look like when the network drops?”
Start with the picture of what goes wrong without it. Imagine Alice taps Send in an elevator. The request reaches the server and commits, but the response never returns. Alice’s app times out and sends the same text again.
Bad design: new id every attempt, order by phone clock. Each retry invents a fresh identity (or none). The server treats each attempt as a new message and sorts bubbles by the phone’s wall clock. Users see two identical bubbles, or a later retry that leaps ahead because clocks are skewed across regions.
Attempt 1: commit message A (wall 12:00:01)
Response lost ↓
Attempt 2: commit message B (wall 12:00:02) ← duplicate bubble
UI sorts by wall clock → wrong or doubled story
Great design: stable client_msg_id + server seq. The phone reuses one client_msg_id for that tap. The server looks up (conversation_id, client_msg_id). If it is found, the server returns the same msg_id and seq. If not, it allocates the next seq, persists ciphertext, then acknowledges. Rendering sorts by seq, then msg_id on ties, not by wall clock.
Attempt 1: client_msg_id=X → seq=17 commit → response lost
Attempt 2: client_msg_id=X → same seq=17 returned
UI shows one bubble in seq order

Figure: At-least-once networks need idempotent sends. The server seq, not the phone clock, owns order.
There is one more detail that interviewers like. A retry can arrive while the first attempt is still being processed, so “look up, then insert” can have its own race, where both attempts find nothing and both insert. The fix is to let the database decide: the row for (conversation_id, client_msg_id) has a unique constraint, or the conversation’s single owner handles one send at a time, so exactly one attempt wins and the other reads the winner’s result.
Scale inside the box: The remembered ids need a retention window per chat, for example a day or two of retries, so the lookup table does not grow forever.
Where it goes next: “You keep saying seq. Who hands out those numbers without two servers handing out the same one?”
“How do you assign the sequence number without two writers colliding?”
The simplest answer is that each conversation has one owner: one machine (or one database partition) is responsible for handing out numbers for that chat, one send at a time. Because the lookup for a repeated id, the next number and the save all happen in that single place, there is no way for two sends in the same chat to receive the same number, and no way for a repeat to slip in between. Many teams put the counter inside the same database row or partition as the messages, so assigning the number and saving the message are one atomic step.
Gaps are fine. If a send fails after taking number 17, the chat can go from 16 to 18 and phones simply sort by seq. Phones only need the numbers to rise, not to be consecutive. If a phone sees a jump, it can ask SyncMessages for what it missed.
The cost is that one very busy chat funnels all its writes through a single owner. That is the right trade for correct order, and it is why a viral group can run a single machine hot, which the Key challenges and Scaling sections come back to.
Where it goes next: “The message is saved. What if telling Bob fails?”
“What if the save works but telling the other phone fails?”
After the commit, the server does three things in order. It looks up where the recipient and each of their devices are connected. If a live connection is healthy, it puts the ciphertext on that connection, respecting a cap on how much can queue up for one slow phone. If the recipient is offline, it hands a generic push (“New message”) to the push service and relies on SyncMessages later. It never puts the decrypted body in the push.
If the commit succeeded but any of that fails, nothing is lost, because the inbox row already exists. The next time Bob’s phone connects or opens the app it asks for everything after its cursor and the screen is repaired. If instead the server had acknowledged before committing, it would have invented a false “sent” tick, and trust collapses. That is why the order is commit, then acknowledge, then notify.
Where it goes next: the interviewer will ask who holds all those live connections, and what happens when one of those machines dies.
“Who owns the live connections, and what happens when a connection server dies?”
Connection servers own the live connections, the heartbeats that check they are healthy, and a small outgoing queue for each phone. The chat service owns durable order and inbox rows. Sticky routing, where a user tends to land on the same connection server, helps keep that user’s session information close, but durability never lives only in the memory of a connection server.
So when a connection server dies, the phones that were connected to it simply reconnect to another one and send their last cursor. They resume from the inbox, with no message lost, because the messages never lived in the dead server’s memory. The risk is not the loss of data but a reconnect storm: millions of phones returning in the same few seconds, which is why phones reconnect with exponential backoff (waiting a little longer after each failed try, with random variation) and why deploys are rolled out gradually.
Where it goes next: “You said the server only sees scrambled bytes. How does that work with several devices?”
“How does encryption work when the server can’t read anything and I have two phones?”
At system-design level, the answer is short and honest. Each device has its own keys. The public half of each device’s key is stored in a key directory on the server, which is safe because public keys only allow scrambling, not unscrambling. When Alice sends, her phone looks up the public keys for each of Bob’s devices and scrambles the message for each of them, so the server stores one ciphertext per recipient device. Only the devices that hold the matching private keys can read it.
Then say what that costs, because it is what the interviewer is really listening for. Push notifications can only say “New message.” The server cannot search the message bodies, so search runs on the phone after it decrypts, or only covers routing facts such as who talked to whom and when. Moderation works through user reports, rate limits and metadata, not through reading chats. If the interviewer pushes further, name forward secrecy, which means old keys are replaced regularly so a stolen key cannot open old or future messages, and then return to inbox, order and fan-out.
Where it goes next: “Now make the group five hundred people.”
“How would you design a five-hundred-member group?”
Copying the message into five hundred inboxes is the simple answer, and it works until it does not. Each copy is a write, and under encryption each copy is also prepared for each member’s devices, so one message in a large group costs hundreds of writes. For hundreds of members, the better model is to store the message once in a shared group log and let each member read with a cursor, the last number they have seen. Sending becomes one write, and reading is spread across the members.
Then keep the model honest. Cap the group size or use announcement mode, where only administrators may post, for broadcast-like traffic. Rate-limit messages per group and chart writes per database partition, so one viral group cannot hot-spot a machine.
Where it goes next: “And how do read ticks work in that group?”
“How do read receipts scale in a large group?”
Delivered means the server handed the message to the recipient’s connection or inbox. Read means the recipient’s app told us it opened the thread, which is reported by the phone and not known independently by the server. In a twelve-person family group, one read event per member is fine. In a five-hundred-person group, one read event per member per message can cost more than the message, so you throttle (limit how often one phone reports), batch (combine reports into one write), show an aggregate such as “read by 42,” or turn granular receipts off. Say which product bar you choose.
Typing and online status are a similar case. They are short-lived signals on the live connection, sent only when the app is in the foreground and merged when repeated, and they are never saved. If you put typing on the durable inbox path at full detail, you create a second product’s worth of writes without making chat more reliable.
Say it in the interview: “One retry-safe send: the phone’s unique id makes repeats harmless, one owner per chat hands out the order number, and the message is saved to the inbox before anyone is told anything. Everything else, the live connection, the push, the cursor, only helps the other phone find out sooner. The cost is a hot owner for a very busy chat, and I would cap the group size and watch writes per partition.”
Hot partitions and connection memory are covered in Key challenges. Save-to-connection latency and queue depth belong in Observability and Bottlenecks and tradeoffs.
If you remember one thing: The deep dive is a conversation. A new id on every retry is the red flag, a stable client_msg_id plus a per-chat seq is the fix, and each answer is something you could have said before they asked.
Key challenges
The deep dive showed that the core send is simple once it is retry-safe and ordered. What makes the real system hard is everything around that step, and these problems show up again and again. Name them early in the interview, because each one is a place the interviewer is likely to push. For each row, say what the user sees if you get it wrong.
| Challenge | What users see if it goes wrong | What to say in the interview |
|---|---|---|
| Encryption + many devices | “Waiting for message” on one phone while another works | Encrypt for each device; key repair and multi-device sync flow |
| Order on network retry | Duplicate bubbles or wrong order | Sort by seq, then msg_id; dedupe client_msg_id on retry |
| Big groups | Lag, failed sends, phone gets hot | State max size; shared log, announcement mode, or rate limits per group |
| Typing indicators | Sluggish app, battery drain | Throttle; send only when app is in foreground; respect hide last seen |
| Video on bad Wi‑Fi | Stuck on one checkmark while text works | Chunked upload with resume; separate timeout and path from text |
Correct order and a durable inbox matter more than which message broker brand you name, and encryption limits what receipts and search can mean on the server.
Scaling the system
Those hard parts get worse as traffic grows, so the next question is what happens when the number of people and messages multiplies. You do not have to wait for the interviewer to ask. Right after you finish the design, say the scaling story yourself, then be ready for the follow-ups.
What you can say upfront: “Inbox storage scales by splitting it across many machines, by chat or by recipient, and I watch writes per partition because one very busy chat can run one machine hot. Connection servers scale by adding more of them by user or region, with a cap on each phone’s outgoing queue, and after any deploy phones reconnect with backoff so they do not all arrive in the same second. Push workers scale on their own and respect Apple’s and Google’s limits. Across regions, order inside a chat is guaranteed by its owner, and I would say honestly that a regional failover can briefly relax it.”
That one paragraph answers most of what follows. Here are the follow-ups, in the order they usually come.
Interviewer: “Traffic doubles next quarter. What do you change?”
You: “Connection servers first, because they hold no important state. The messages live in the inbox, so I add machines and phones spread across them. Then I check the storage side: how many partitions, how full they are, and which chats are busiest. If there is headroom, nothing changes there.”
Interviewer: “How do you shard the inbox?”
You: “Either by chat id, so one conversation’s messages and order live together, or by user id, so each person’s inbox lives together. By chat is simpler for order and group logs, and it works until one chat becomes huge. By user is simpler for catch-up, and it makes group sends write into many partitions. I pick one, say why, and watch traffic per partition either way.”
Interviewer: “One live-event group is getting hundreds of messages a second. What happens?”
You: “That is the case sharding does not solve. The chat’s order has one owner, so every message in that group goes through the same partition, and typing signals make it worse. I would rate-limit messages per group, move very large audiences to announcement mode or a shared log with cursors, merge typing events on the connection server, and cap the group size. The cost is that a very large group is not a free-for-all chat.”
Interviewer: “The connection fleet restarts for a deploy. What could go wrong?”
You: “Every phone loses its connection at once and reconnects at once, which can overload the servers that just came back. So I roll deploys out in small slices, drain connections slowly, and have phones reconnect with exponential backoff and random variation. They resume from their cursor, so nothing is lost, but the dashboards can look healthy while users see ‘connecting…’.”
Interviewer: “Do you need more than one region?”
You: “Often yes, for speed and for data-residency rules. Order inside a chat stays authoritative through its owner. Strict order across regions during a failover is hard, so I would say plainly that a failover can produce a brief inconsistency until phones sync, and that I am choosing availability over perfection for a few seconds.”
Interviewer: “How do push notifications scale?”
You: “Push workers scale separately from chat writes, because they depend on Apple’s and Google’s rate limits and not on our database. I delete dead device tokens aggressively so we do not retry forever, and I treat push as a hint, since the inbox already holds the message.”
If you remember one thing: Connection servers scale by adding machines; the inbox scales by splitting data, and one very busy chat needs its own limits.
Failure handling
Scaling answers “what if there is more traffic.” This section answers “what if something breaks.” Again, say it first. Failures in a chat system come from flaky phone networks, from connection servers, from push providers and from whole regions, and what impresses an interviewer is that the answer was decided before the failure happened.
What you can say upfront: “Every failure has a decision made in advance. The sender’s tick only appears after the inbox row is saved, so nothing the user was told about can be lost. If the connection drops mid-send, the phone retries with the same unique id and the server returns the same result. If the save works but telling the other phone fails, the inbox already has the message and the phone catches up from its cursor. A push that is accepted but never shown only delays the message until the app opens. If a region fails, I fail over and I am honest that order can briefly glitch. A real outage is ‘cannot send at all’ or ‘messages disappear,’ and saving before acknowledging is what prevents the second one.”
The follow-ups are about the details of those decisions.
Interviewer: “The connection drops in the middle of a send. What does the user see, and what does the system do?”
You: “The user sees a spinner on the message. The phone reconnects with backoff and resends with the same
client_msg_id. If the first attempt reached the server, it returns the saved result, and if it did not, it saves it now. Either way there is one bubble.”
Interviewer: “How do you know a message was saved if you are not sure?”
You: “I fail closed on ambiguity for the sender: I do not show ‘sent’ until the inbox row is committed. Users will tap send again in the meantime, and duplicate detection makes that safe.”
Interviewer: “A video upload stops halfway. What happens?”
You: “The message shows one gray tick on the chat while the video is still uploading, and the upload resumes from where it stopped, using the resumable upload on object storage. Broken partial files expire after a while. A broken upload never blocks other messages in the chat, because they take a different road.”
Interviewer: “Push says ‘delivered’ to Apple but the user saw nothing.”
You: “That is normal. Do Not Disturb, battery saver or a stale token can hide a banner. The message waits in the inbox until the user opens the app, and the phone syncs from its cursor. I measure push failures separately from in-app display problems, and I never treat push as proof of reading.”
Interviewer: “A whole region goes down.”
You: “I fail over to another region and phones reconnect there and resume from their cursors. Users see delay, and possibly a brief order glitch, because strict order across regions is expensive. I say that out loud as a deliberate trade for staying available.”
| Scenario | What the user sees | What to build |
|---|---|---|
| Connection drops mid-send | “Sending…” spinner on phone | Reconnect with backoff; resend same client_msg_id |
| Video upload stops halfway | Stuck on one gray tick | Resume upload; expire broken partial blobs |
| Push accepted but app closed | Message late until user opens app | Inbox sync on open; normal, not a bug |
| Whole region unavailable | Delay, possible brief order glitch | Fail over; admit cross-region limits honestly |
If you remember one thing: Saving to the inbox before the tick is what prevents loss; slow delivery is a different, much milder class of problem.
Observability (what you measure)
Failure handling decides what to do when something breaks. Once the system is live, you also need a way to tell whether it is healthy, and a green success rate on the send call alone will lie—users can still stare at a spinner while that number looks fine. Measure the path people actually feel:
| Signal | Why it matters |
|---|---|
| Inbox commit success / latency | Delivery promise to the sender |
| Save-to-connection p99 (how long the slowest 1 in 100 takes) | Time until an online peer can decrypt |
| Push accept vs in-app render lag | Separates vendor delivery from UI truth |
Writes per partition / top conversation_id | Catches viral-group hot shards |
| Connection queue depth / disconnect reason | Memory pressure before CPU looks busy |
Dedup hit rate on client_msg_id | Confirms retries are common and working |
Alert when save-to-connection or partition write skew moves, not only when API 5xx jumps.
Security and abuse
A chat system holds private conversations, so after failures and metrics the next question is how to keep the product honest. Check that every send and sync comes from a signed-in user, and check that the person is a member of the chat before writing into it. Under end-to-end encryption the server never needs readable message text for routing—treat accidental logging of that text as a privacy incident. Cap how fast someone can send, request signup codes, or create groups; slow typing and “online” signals so bots cannot flood the connection servers. Abuse handling should use small routing facts (who talked to whom), graph signals and user reports, not “read the chat from the database.” Paths to rotate keys across several devices and to revoke a stolen session matter as much as encrypting the connection on the wire (TLS). Media upload links must be short-lived and limited in scope so one leaked link cannot empty object storage.
Cost awareness
Security keeps the product honest; cost is the other quiet constraint interviewers like to hear. Every open connection costs memory; every “copy into each member’s inbox” group multiplies storage and writes by member count; every media file stored forever dominates disk more than text ever will. Choosing a shared history for huge groups trades write cost for more read traffic and hotter database slices. Generic push at billions of users is an invoice line with Apple and Google, so delete dead device tokens aggressively. Search indexes built on the phone cost battery and storage so the server can stay ignorant of readable text; that trade is intentional, not free. Archive and “delete for everyone” policies are cost features as much as product features.
Green Dashboard. Angry Users. The Production Mindset System Design Interviews Expect.
Everything so far describes how the chat system should work on a good day. This section is about what happens on a bad day, told as short stories from the people who carry the pager (the on-call engineers). Textbooks teach boxes and arrows. On-call teaches this: the graph can look fine while users are furious.
SendMessage returning success is not the same as the friend seeing the bubble. Apple marking a push “delivered” is not proof the human read the chat. Interviewers love this gap because it separates diagram-only answers from people who have thought about launch day.
Three nightmares that look “healthy” on a dashboard:
| What the dashboard shows | What is actually happening |
|---|---|
| Chat API success rate and database latency look normal. | Users stare at “connecting…” for seconds because connection servers are saturated or reconnecting after a deploy. |
| Push provider reports delivered to Apple or Google. | The user saw no banner (Do Not Disturb, battery saver, or a stale token), and the message waited in the inbox until they opened the app. |
| Global QPS and CPU are flat on chat workers. | One viral group hammers a single database partition; sequence allocation and typing indicators make the hot shard worse. |
Track the whole path (inbox commit → fan-out → live connection or push → decrypted screen), not one green metric in the middle.
Scene 1: APIs green, chat feels slow
The moment: It is evening peak in a large city. People are sending the usual flood of messages, and support starts hearing that messages take a long time to show one checkmark, then two. The dashboard for the send API shows healthy latency.
The trap: Long-lived connections fail differently than one-off web requests. Teams often alert when the send API returns server errors, but not on the time from “saved in the inbox” to “first byte on the friend’s open connection.” New encrypted handshakes, pauses while the server cleans memory, and herds of phones reconnecting after a deploy can hurt chat without moving the “API success” graph.
What to build:
- Measure save-to-connection latency end to end, not only
SendMessage200. - Load-test connection pools before peak; rehearse deploys with reconnect backoff so phones do not amplify an outage.
- Remember: queue lag zero does not mean the user saw the message on screen.
Scene 2: Push delivered, user saw nothing
The moment: A customer writes to support saying they never got notified about a message from a friend. The push provider’s dashboard shows the pushes were delivered.
The trap: Push only tries to wake the app. Under end-to-end encryption, the server often cannot put message text in the banner. “Delivered to Apple” is not “the person read the thread,” and it is not even “the app unscrambled the bytes and drew the bubble.”
What to build:
- Treat push as help, not proof of human attention.
- Drive unread state from synced inbox rows on the device.
- Measure push failures separately from in-app rendering bugs.
Scene 3: One viral group melts one shard
The moment: A large live event opens a public group, and thousands of people start writing in it at once. Messages in that group fail or lag, and one conversation_id is driving hundreds of writes per second.
The trap: Splitting the database by chat id works until one chat becomes enormous. Correct order inside a conversation needs a single owner handing out sequence numbers, which is right for the user experience and a hot slice of the database at scale. Typing indicators multiply the writes.
What to build:
- Rate-limit messages per group; use announcement mode for broadcast-scale channels.
- Cap group size or move stadium traffic to a different product surface.
- Chart writes per partition, not only global message QPS.
Scene 4: CPU calm, memory spikes, random disconnects
The moment: The team that runs the connection servers sees healthy-looking clients being disconnected at random. Memory on the servers climbs while CPU looks bored.
The trap: Each open connection keeps a small outbound queue of bytes waiting to be sent. A phone on slow Wi‑Fi or in the background reads those bytes slowly; frames pile up until the server must drop or disconnect to protect its memory.

Figure: A flat CPU graph can hide per-connection queues eating memory.
What to build:
- Cap queue depth per connection; merge repeated typing events.
- Disconnect gracefully with a resume cursor so phones do not replay full history blindly.
- Alert on queue depth and time from save to first byte sent.
In the interview: Pick one scene. Tell it like a short story: what users felt, what metric lied, what you would ship next. End with: you do not need global order—one rising seq per chat is enough, and say why.
Bottlenecks and tradeoffs
Those stories all trace back to a small number of choices where you cannot have both things you want. Interviewers listen for whether you can name the tension, say what breaks, and pick a side on purpose. Here are the three that matter most for a chat system.
Encryption vs search and moderation
Decision 6 already forced the privacy choice; the tradeoff is what that choice costs day to day. Users want private chats; the product also wants search, abuse detection and support tooling.
What “fair” can mean for privacy:
- True E2E: the server stores ciphertext only; search and moderation use metadata, reports or client-side indexes.
- Metadata search: who talked to whom, when, message size, never the body.
- Product honesty: name what you give up on the server; do not pretend support can “just read the chat.”
What to build: A client-side search index after decrypt, or limited server tools on metadata. Abuse pipelines on rate, graph and user reports, not logging plaintext “for debugging.”
In the interview: Say explicitly what the server cannot do if chat is E2E. Weak answers claim “we’ll index messages for support.”
Same order on every device vs speed and cost
The scenes also showed phones catching up from a cursor; that leads to another tradeoff. Users want the same thread on phone, tablet and web, but perfect instant sync everywhere costs extra round trips.
What breaks: Brief moments where two devices show slightly different order until SyncMessages catches up.
What to build: Many products accept good enough order that converges in seconds; stricter linearizability costs latency and engineering.
In the interview: Pick a bar, “good enough for chat” or “strict same order everywhere,” and justify it with seq and sync cadence.
Small groups vs stadium groups
Decision 7 and Scene 3 already put a number on group size; here is the tradeoff in one place. Copying each message into every member’s inbox feels simple for family chats; at stadium scale, the writes explode.
What breaks: One viral group overloads one database partition.
What to build: State max group size; switch to shared log + cursor or announcement mode above that limit.
In the interview: Group size is not a footnote. It picks write amplification vs read amplification. Name your cap and model aloud.
Interview tips
You now have a full design and the tradeoffs behind it, so the last thing to practice is defending it when the interviewer pushes back. These five are the pushes that come up most after you finish the design:
Two phones show different order — wall clocks are not the source of truth
In the room
You might say: “Messages in a chat are ordered by the timestamp the phone attaches when you send them.”
Interview Push: “Alice in London and Bob in Tokyo both send at ‘the same second’ on phones with bad clocks. What order do users see?”
Land here: The server assigns a rising sequence number (
seq) inside each chat, not one global order for the whole planet. If two messages somehow tie, break the tie with the server’s message id. Never sort by the phone’s wall clock: clocks disagree, networks reorder packets, and a later retry can look “earlier” on a skewed device clock. When two devices briefly disagree after reconnecting, a catch-up call with the last cursor (SyncMessages) brings them onto the sameseqorder. Global total order across every chat on earth is not the goal; order inside one conversation is.
The elevator retry — at-most-once loses messages users care about
In the room
You might say: “At-most-once delivery is simpler: if the phone never retries, we never show a duplicate.”
Interview Push: “A user taps send in an elevator, the connection drops, and they tap send again. Do you drop one copy or show two bubbles?”
Land here: Mobile networks often deliver the same send more than once (at-least-once delivery). Dropping the second attempt to “avoid duplicates” can lose the only successful save when the first response never returned. Keep a stable unique id for that one tap (
client_msg_id); the server returns the same message id and sequence number when that id arrives again. The screen shows one bubble; order stays tied toseq, not to how many times the phone retried. The elevator picture from the dig is the whole story: the reply was lost, the phone reused the same id, and the server handed back the same sequence number.
Push carries the message — encryption says otherwise
In the room
You might say: “When the friend’s phone is offline, we send a push notification that contains the chat text.”
Interview Push: “If messages are end-to-end encrypted, what appears on the lock screen?”
Land here: Usually the banner only says “New message”, not the chat body. The durable row already lives in the inbox on the server; the app fetches the scrambled bytes and unscrambles them after opening (or over the live connection). Push is a wake-up call. Apple or Google accepting a notification is not proof the person saw the thread, and it is not how scrambled bytes become readable text.
We log message text for support — that breaks E2E
In the room
You might say: “We log full message content so engineers can debug faster when a user reports a problem.”
Interview Push: “What can the server legally and technically know if chat is truly E2E?”
Land here: The server moves scrambled bytes (ciphertext) plus small routing facts such as sender, size and time—not readable text. For abuse, use send-rate caps, who-sent-to-whom signals and user reports. Do not claim support can open a thread from the database. If you need structured debugging, store reason codes on delivery failures, not chat bodies. Saying “we’ll add searchable plaintext later” quietly undoes the privacy promise you opened with.
Typing updates to everyone — write storms at billion-user scale
In the room
You might say: “We broadcast the ‘typing…’ signal to all of a person’s contacts in real time.”
Interview Push: “A billion daily users open keyboards. What happens to your write load?”
Land here: Slow how often you send typing and online signals, and send them only while the app is in the foreground. Combine or sample updates, and merge repeated “typing…” events on the connection server so memory does not grow forever. Respect a user’s “hide last seen” setting. Presence is optional polish, not on the same urgency path as sending a real message. If you save every typing signal on the durable inbox path, you invent a second product’s worth of writes without making chat more reliable.
Red flags to avoid without correcting yourself: “Order by phone timestamp,” “push is the message,” “log plaintext for support,” “copy every send into fifty thousand inboxes,” “at-most-once so we never dedupe.”
After each interview push, close with one real part of the design you would build (seq, client_msg_id, inbox sync, ciphertext storage or throttled presence), not “we’ll scale it later.”
What should stick
After working through the requirements, the decisions and the depth sections, you should be able to explain:
- Two paths. Short text, ticks and presence travel on a fast lane; photos and video upload on a separate truck lane through file storage and a content network, and the chat only carries a pointer.
- Inbox before push. Saving the message in the recipient’s inbox on the server is the delivery promise; a push notification only tries to wake the app.
- Per-chat sequence plus tap id. The server’s rising number per conversation beats phone clocks for order, and a unique id for that one tap makes a network retry show one bubble instead of two.
- Groups pick the model. Small groups can copy each message into every member’s inbox; large groups need a shared history with cursors, or a stated size cap.
- Ops beyond a green send call. Watch time from inbox save to the peer’s connection, writes on the hottest database slice, and how deep each phone’s outbound queue grows—not only whether the send API returned success.
Tell it in the room: “I split text and media. One send: encrypt, dedupe with client_msg_id, save to inbox with seq, notify the online peer on the live connection or the offline peer with a generic push. Groups: I state max size and pick copy-on-send or shared log. E2E means ciphertext only on the server, so push cannot quote the message.”
Quick check before a mock: (1) What does one gray tick promise? (2) Why is the phone clock wrong for ordering? (3) What makes a retry safe? → (1) the inbox row is saved (2) clocks disagree and retries look old or new (3) a stable client_msg_id that the server dedupes.
Practice next: Open Design WhatsApp on Practice System Design and run a timed attempt using clarifying questions and one message story, without peeking until you have a draft on the board.
Key Takeaways
- A tick means the message is saved in the recipient’s inbox on the server, not that a notification was shown; push is only a wake-up hint.
- Make retries harmless with a unique
client_msg_idper tap, and let the server return the samemsg_idandseqwhen it sees that id again. - Order each chat with a server-assigned
seqper conversation; phone clocks cannot be trusted and global order is not needed. - Keep photos and video on a separate upload road through object storage and a content network; the chat carries only a
media_idpointer. - End-to-end encryption means the server stores ciphertext: push text is generic, search runs on the phone, and moderation uses reports and metadata.
- State a maximum group size: copy per member below it, a shared log with cursors above it, and throttled or aggregated receipts in big groups.
- Watch save-to-connection latency, writes per partition and connection queue depth, not only the send API’s success rate.
Related Topics
- How to Prepare for System Design Interviews - Multi-week study plan, pacing, and prep mistakes
- Notification Service System Design - Push delivery, device tokens, and why “delivered to Apple” is not “user read”
- Facebook Feed System Design - Contrast read fan-out and ranking with per-thread chat order
- Distributed Cache System Design - Presence maps and hot key patterns for online-user routing
- Message Queues - Where async workers fit after the inbox commit (not always on every text send)
- WebSockets and Real-Time - Long-lived connections, heartbeats, and reconnect storms
Quick revision notes
The problem in your own words. A chat system carries a message from one person to another or to a group, keeps it safe if the other phone is offline, and does so without showing duplicates, without mixing up the order and without the server reading the text. The hard part is that phones retry on bad networks and clocks disagree, so the server, not the phone, has to decide identity and order.
The design in one breath. The phone encrypts the message and sends it with a unique id for that tap. The server ignores a repeat of that id, assigns the next sequence number for the chat, and saves the scrambled message to the recipient’s inbox before telling the sender it is stored. Online phones get it over their live connection, offline phones get a generic push, and any phone can catch up from its last cursor. Photos and video go by a separate upload road and appear as a small pointer.
The choices you must be able to defend. Save before acknowledging, so the tick is a promise. At-least-once delivery with deduplication instead of at-most-once. A per-chat sequence number instead of phone time. Copy per member for small groups, a shared log with cursors for large ones, with a stated size cap. Throttled or aggregated receipts in big groups, and typing signals that are never saved.
The numbers worth remembering. About a billion active people and tens of billions of messages a day, which is roughly 580K sends a second on average for 50 billion a day (an estimate). Hundreds of millions of open connections, about 500 machines at 200K each for 100 million. A 256-member group copies a message 256 times and a 50,000-member group 50,000 times.
The mistakes that cost points. Ordering by the phone clock, treating push as the message, logging plaintext for support, copying every send into a huge group’s inboxes, picking at-most-once delivery, and putting photos on the same path as text.
Frequently asked follow-ups
- How does end-to-end encryption work with server routing?
- How do you order messages across bad clocks and retries?
- How do large groups deliver one message to every member without overloading storage?
- How do offline users catch up?
- How is this different from designing a social feed?
Deep-dive questions and strong answer outlines
Walk through sending one text message in a 1:1 chat.
I would start by asking encryption depth and group size limits. For one text: Alice's phone encrypts the body, generates a unique client_msg_id, and calls SendMessage with ciphertext and metadata. The server checks whether this client_msg_id already exists for the chat—if yes, return the same msg_id (idempotent retry). Otherwise assign msg_id and rising seq for that chat, persist ciphertext to Bob's inbox, then notify: if Bob has an open socket, push on the connection; if offline, save first and send a generic push ("new message") that cannot include plaintext under E2E. Bob's phone decrypts and sends delivered/read acks when the UI allows.
How do read receipts scale in large groups?
Delivered means the server handed the message to the recipient's connection or inbox. Read (two blue ticks) means the recipient's app reported they opened the thread—that is client-reported, and encryption limits what the server can infer. In a twelve-person family group, per-member read events are fine. In a five-hundred-person group, one read event per member per message explodes write load—throttle, batch, show "read by 42" aggregates, or disable granular receipts. Say which product bar you pick.
How would you design a five-hundred-member group?
Copy-on-send—writing one row per member per message—works for small groups but fails at stadium scale. For hundreds of members I would store messages once in a shared group log and have each member read with a cursor (last seen seq). Cap group size or use announcement mode (only admins post) for broadcast-like traffic. Rate-limit messages per group and chart traffic per database partition so one viral group cannot hot-spot a shard.
What can the server do if chat is end-to-end encrypted?
The server routes ciphertext and metadata (sender, recipient, timestamp, size)—not readable message text. Push notifications are usually generic. Keys stay on devices; the server may store public keys per device for setup. Abuse handling uses rate limits, reports, and metadata—not reading chats from the database. Search on the server is limited unless the phone builds a client-side index.
How do offline users receive messages?
The message is durable in the inbox on the server before the sender gets acknowledgment—that is the delivery promise. Push tries to wake the app but may fail (Do Not Disturb, dead token, battery saver). When the user opens the app, SyncMessages(cursor) pages through everything after their last cursor until caught up. Push means "please open the app," not "the human definitely read it."
How is messaging different from designing a news feed?
Feeds optimize read fan-out and ranking; chat optimizes per-thread order, low-latency delivery, and strong privacy. Feeds tolerate seconds of delay; chat users notice hundreds of milliseconds. Feeds often store plaintext for ranking; E2E chat stores ciphertext and uses per-chat seq, not a global timeline score. Connection servers and inbox sync matter more than a pull-based feed API.
AI feedback on your design
After a practice session, InterviewCrafted summarizes strengths, gaps, and interviewer-style expectations—similar to a written debrief. See a static example report, then practice this problem to get feedback on your own answer.
FAQs
Q: Do I need to explain Double Ratchet cryptography in detail?
A: Usually no unless the interviewer pushes. At system-design depth, say phones establish keys, messages are encrypted on the client, and the server sees only ciphertext. If they want more, name forward secrecy (old keys cannot decrypt future messages after compromise) and multi-device key distribution—then return to inbox, seq, and fan-out. Do not spend twenty minutes on crypto math while ignoring ordering and offline sync.
Q: Should Kafka sit on the hot send path?
A: Many production chat systems use custom inbox storage plus long-lived sockets for sub-100ms signaling—not a heavy queue on every send. Queues still help for async work: push delivery, abuse scoring on metadata, analytics, and media processing. In the interview, say: commit to inbox first, notify online peers synchronously or near-sync, offload everything else async. Pick Kafka if they ask about cross-service decoupling, not as the default answer for every text message.
Q: How does phone-number signup fit the architecture?
A: Signup is a separate identity path: verify SMS or voice OTP, create user id, register device tokens for push, upload initial public keys. It is not on the hot message path. Mention it in one sentence unless they ask about fraud (rate-limit OTP, detect SIM swap abuse). The messaging design stands on inbox, connections, and encryption—not on how the account was created.
Q: How do you scale connection servers to billions of users?
A: Shard connection servers by user id or region. Use sticky routing so the same user tends to land on the same box for stateful features. Cap outbound queue size per socket so one slow phone does not exhaust server memory. Plan for thundering herd reconnects after deploys with backoff. Measure open sockets, queue depth, and time from DB save to first byte on the wire—not only HTTP API success rate.
Q: Can the server search encrypted messages?
A: Not the message body if chat is truly end-to-end encrypted. Search happens on the phone (client-side index built after decrypt) or you accept limited server-side search on metadata only. Moderation relies on user reports, rate limits, and who-sent-to-whom graphs—not reading every chat from logs. Name this tradeoff explicitly; pretending the server can "just index messages" fails the privacy question.
Q: What is the difference between delivered and read?
A: Delivered (two gray ticks) usually means the server successfully placed the message on the recipient's connection or in their inbox. Read (blue ticks) means the recipient's app reported they viewed the thread—the server forwards that signal but does not independently know the user read with their eyes. Under E2E, notification text cannot quote the message. In interviews, separate server delivery from human attention and from push notification accepted by Apple/Google.
Practice interactively
Open the practice session to use the canvas and stages, then review AI feedback.