Requirements and estimates
- One-to-one and small group chats (up to a few hundred members).
- Messages delivered in order within a conversation, with sent / delivered / read receipts.
- Online presence and last seen; push notifications for offline users.
- History synced across a user's devices.
- Low latency (under ~200 ms when both users are online) and no lost messages.
50M daily users × 40 messages/day = 2B messages/day ≈ 23k/s (peak ~70k/s)2B × 200 bytes ≈ 400 GB/day of message dataconcurrent connections ≈ 10M+ open WebSockets
Real-time connections
HTTP request/response can't push messages to a client. Options: polling (wasteful), long polling (better), Server-Sent Events (server → client only), and WebSockets (full duplex over one persistent connection, the usual choice).
Chat servers hold the WebSocket connections and are stateful: a user is connected to one specific server. A session registry (Redis: user → server) records where each user is, so a message can be routed to the right chat server.
Sending a message
- The sender's chat server receives the message and assigns it a per-conversation sequence number (or a time-ordered id).
- It persists the message first, then acknowledges to the sender ("sent" tick). Persist before ack means no message is lost.
- It looks up the recipient in the session registry. If online, it forwards the message to that user's chat server (directly or via a pub/sub channel), which pushes it down the WebSocket.
- The recipient's device acks, and a "delivered" receipt flows back. If offline, a push notification is sent and the message waits in storage until the device reconnects and syncs.
Storing messages
The access pattern is "write a message; read the latest N messages in a conversation; page backwards". A wide-column store such as Cassandra or HBase fits: partition key conversation_id, clustering key message_id descending. Each conversation's messages sit together in order, and writes scale horizontally.
Each device remembers the last message id it has seen per conversation; on reconnect it asks for everything after that, which also syncs multiple devices.
Group chats fan out on write for small groups (deliver to each online member). Very large channels switch to fan-out on read: members fetch from the channel when they open it.
Presence
Clients send a heartbeat every ~30 seconds; a user is online while heartbeats arrive and offline after a few are missed (so a flaky network doesn't flap the status). Presence changes are published only to the user's contacts who are online, and large groups don't show live presence because the fan-out would be enormous.
Delivering a message
Alice sends "hi" over her open WebSocket to chat server 1.
See it animated
Interview questions
Try answering out loud first, then open the model answer.
Q1Why WebSockets instead of polling?beginner
Polling sends a request every few seconds whether or not there's a message: most requests are empty, and latency equals the poll interval. A WebSocket keeps one connection open and the server pushes messages the moment they arrive, with little overhead per message.
Q2How do you route a message to a user connected to a different server?intermediate
Keep a session registry mapping user id to the chat server holding their connection. The sender's server looks it up and forwards the message to that server, directly or through a pub/sub channel per server or per user, which then pushes it down the socket.
Q3How do you guarantee ordering within a conversation?advanced
Assign a monotonically increasing sequence number per conversation at the server (for example an atomic counter keyed by conversation), store messages by it, and have clients sort and detect gaps by it. Client timestamps can't be trusted because device clocks differ.
Q4How do you avoid losing messages?intermediate
Persist before acknowledging the sender; have clients retry unacknowledged sends with a client-generated message id so retries are deduplicated; and have recipients sync from their last-seen id on reconnect, so anything missed while offline is fetched.
Q5Which database would you use for messages, and why?advanced
A wide-column store like Cassandra or HBase: huge write throughput, horizontal scaling, and data laid out by partition key in sorted order, which matches "latest messages in conversation X". Users and group membership can stay in a relational database.
Quick check
0 of 3 answered