Requirements and estimates
- Users post (text, images) and follow other users.
- The home feed shows recent posts from people you follow, newest or ranked first, with infinite scroll.
- Feed loads in under ~200 ms; a new post can take a few seconds to appear (eventual consistency is fine).
300M daily users, 10 feed loads each = 3B feed reads/day ≈ 35k/s (peak ~100k/s)1 post per user per 5 days = 60M posts/day ≈ 700 writes/saverage 200 followers; some accounts have 100M
Fan-out on write vs read
| Fan-out on write (push) | Fan-out on read (pull) | |
|---|---|---|
| When a user posts | Insert the post id into every follower's feed cache | Just store the post |
| When a user reads | Read their precomputed feed list: very fast | Fetch recent posts from everyone they follow and merge: slow |
| Cost | Huge for accounts with millions of followers | Repeated work on every load |
| Wasted work | Feeds of inactive users are updated anyway | None |
Real systems use a hybrid: push for normal accounts, pull for celebrities. At read time the service takes the user's precomputed feed and merges in recent posts from the few celebrities they follow. Merging k sorted lists is a heap problem.
Components
- Post service stores posts (sharded by post id or author) and publishes a post-created event.
- Fan-out workers consume the event, look up followers (graph service) and append the post id to each follower's feed list in a cache such as Redis, trimmed to the latest ~500 ids.
- Feed service reads the feed list, merges celebrity posts, hydrates ids into full posts from the post cache, applies ranking and filtering, and returns a page.
- Media lives in object storage behind a CDN; the feed only carries URLs.
Ranking and pagination
Chronological feeds sort by time. Ranked feeds score candidates (affinity with the author, engagement, recency, content type) and sort by score; a simple start is score = engagement weight × time decay. Use cursor pagination so new posts arriving at the top don't shift pages.
Hybrid fan-out
A user with 300 followers posts. The post is stored and a post-created event is published.
See it animated
Interview questions
Try answering out loud first, then open the model answer.
Q1Fan-out on write or on read?intermediate
Push makes reads cheap, and reads far outnumber writes, so it's the default for most users. But posting from an account with 50 million followers would mean 50 million cache writes, so for those accounts pull at read time instead. The hybrid gets fast reads without write storms.
Q2What is the celebrity (hot user) problem?intermediate
A few accounts have enormous follower counts, so pushing their posts to every follower is slow and expensive, and it delays their posts from appearing. The fix is to skip fan-out for them and merge their recent posts into followers' feeds when the feed is read.
Q3How would you merge posts from many followed accounts efficiently?advanced
Each source is already sorted by time, so it's a k-way merge: put the head of each list in a max-heap keyed by timestamp (or score), pop the top, push that source's next item, and stop after one page. That's O(page × log k).
Q4Why store post ids rather than full posts in feed caches?intermediate
A post appears in many feeds; storing ids keeps each feed small and means edits or deletions happen in one place. The feed service hydrates ids from a post cache in a batch request.
Quick check
0 of 3 answered