Design Case Studies · Lesson 13

Design: Social Feed API

Every tweet from a 50-million-follower celebrity has to land in 50 million inboxes. How you route that write — and when you give up on pre-computing it — is the central architectural decision in the social feed problem.

1 — Requirements

Twitter-style systems look deceptively simple from the outside: post a message, follow people, read a feed. The constraints underneath are what make this a canonical distributed systems case study.

Functional requirements

• Post a tweet: text up to 280 characters, optional media attachment, optional reply-to reference. Acknowledged synchronously — the post must exist once the API returns 201.
• Follow / unfollow: create or remove a directed follow relationship between two users. The requester's home timeline should reflect the change on the next read.
• Home timeline: a reverse-chronological feed of posts from the accounts a user follows. Cursor-paginated; must return results within 100 ms at p99.
• User timeline: all posts from a single account. Out of scope for this lesson but structurally similar.

Non-functional requirements

• Scale: hundreds of millions of active users. Timeline reads vastly outnumber writes — roughly 100:1. The system is read-heavy, so read latency is the budget that matters.
• Hot-spot risk: a single celebrity posting generates a write fan-out proportional to their follower count. A naïve fan-out-on-write would collapse under a 50M-follower account posting once.
• Consistency: strong for the write itself — a 201 means the tweet exists. Eventual for timeline delivery — seeing a post 1–2 seconds after it was created is acceptable. A home timeline is not a financial ledger.
• p99 targets: timeline read under 100 ms; write acknowledged under 500 ms.

2 — Design decisions

The fan-out question

Think of a newspaper printing plant. One strategy: at press time, print a personalised edition for every subscriber and drop it in their mailbox — so when they wake up, it's already there, no wait. That is fan-out on write. The other strategy: print one master edition, and at the newsstand, cut custom pages for each reader on demand. That is fan-out on read.

Fan-out on write is fast to read — the timeline cache is pre-populated, so a GET is a single cache lookup. But writing is expensive: every post for an account with N followers triggers N cache prepend operations. For a celebrity with 50 million followers, that is 50 million writes per tweet. The cache tier cannot absorb that synchronously without cascading failures.

Fan-out on read is cheap to write — the tweet lands in one place (the author's post list). But reading is expensive: to build a timeline for a user who follows 500 accounts, you fetch 500 post lists, merge them by timestamp, and paginate. That merge cost grows with the social graph, and p99 latency for users who follow many accounts can blow past the 100 ms budget.

Hybrid routing: the practical answer

Neither pure strategy survives contact with a real social graph. The production approach is a hybrid keyed on follower count:

• Normal users (≤ ~10 000 followers): fan-out on write. On every post, a worker prepends the tweet ID to each follower's cached timeline list. Reads are a single cache fetch. Write amplification is bounded and manageable.
• Celebrity accounts (> ~10 000 followers): skip write fan-out entirely. The tweet goes only into the celebrity's own post list. No follower caches are touched at write time.
• At read time: the timeline service fetches the user's precomputed cache (from normal fan-out) and separately fetches the recent post lists of any celebrities the user follows. It merges these two streams and returns a unified, sorted result.

This caps write amplification at O(followers) for accounts below the threshold — and at O(1) for accounts above it — while keeping read latency bounded by a small, predictable number of celebrity fetches per read.

Cursor pagination (as-03)

Timelines are live: new tweets arrive between page requests. Offset pagination breaks on live data — offset=20 on your second request skips 20 rows counting from the current head, which has shifted since your first request, producing gaps or duplicates. A cursor encodes an opaque position (typically a tweet ID or timestamp) that is stable regardless of insertions ahead of it. The server uses the cursor to resume from exactly where the client left off. Never expose offsets on a feed endpoint.

Timeline caching (rel-07)

Each user's precomputed timeline lives as a Redis list — a sorted sequence of tweet IDs. On fan-out write, the worker prepends to the list. The list is capped at the last N entries (e.g. 800) to bound memory per user. TTL is ~24 hours; a cold read (cache miss) reconstructs from the database, which is expensive and should be rare. A cache hit is the overwhelmingly common path and takes roughly 2 ms.

Eventual consistency is the right trade-off

Fan-out is async: a post enters a queue, and workers process followers in batches over the next few seconds. This means a follower's timeline may lag 1–2 seconds behind a real-time post. That lag is acceptable — a home feed is not a financial ledger, and no safety or legal consequence follows from seeing a tweet a second late. The trade-off buys dramatically lower write latency and vastly simpler fan-out infrastructure.

3 — The API model

Post a tweet

POST /v1/tweets
Authorization: Bearer <token>
Content-Type: application/json

{
  "text": "Cursor pagination prevents ghost items on live feeds.",
  "reply_to": "tweet_xxx"    // optional — omit for top-level post
}

HTTP/1.1 201 Created
{
  "id":         "tweet_abc123",
  "author_id":  "usr_42",
  "text":       "Cursor pagination prevents ghost items on live feeds.",
  "reply_to":   null,
  "created_at": "2025-11-14T09:04:00Z"
}

The 201 confirms the tweet is durably stored. Fan-out to followers happens asynchronously — the response does not wait for it.

Read the home timeline

GET /v1/timeline?cursor=&limit=20
Authorization: Bearer <token>

HTTP/1.1 200 OK
{
  "tweets": [
    {
      "id":         "tweet_abc123",
      "author_id":  "usr_42",
      "text":       "Cursor pagination prevents ghost items on live feeds.",
      "created_at": "2025-11-14T09:04:00Z"
    }
    // …up to 20 items
  ],
  "next_cursor": "eyJ0d2VldF9pZCI6InR3ZWV0X2FiYzEyMyJ9",
  "has_more":   true
}

# Next page — pass the opaque cursor, not an offset
GET /v1/timeline?cursor=eyJ0d2VldF9pZCI6InR3ZWV0X2FiYzEyMyJ9&limit=20

Follow a user

POST /v1/follows
Authorization: Bearer <token>
Content-Type: application/json

{
  "target_user_id": "usr_99"
}

HTTP/1.1 201 Created
{
  "follower_id":  "usr_42",
  "followee_id":  "usr_99",
  "followed_at":  "2025-11-14T09:10:00Z"
}

Unfollow a user

DELETE /v1/follows/usr_99
Authorization: Bearer <token>

HTTP/1.1 204 No Content

4 — Evaluation & latency budget

Write amplification analysis

Without the celebrity exemption, a single post from the largest account would require 50 million cache writes. Even at 10 µs per write, that is 500 seconds of cumulative cache work — serialised. The exemption collapses this to a single write, deferring the merge cost to read time, where it is bounded and parallelisable.

Read latency budget — target p99 < 100 ms

The real write-path bottleneck is fan-out queue lag. If the async worker falls behind, readers fall back to DB reconstruction for newer posts, spiking p99. Monitoring queue depth and consumer throughput is as important as monitoring cache hit rate.

Under the hood: the core mechanism

The timeline cache is a Redis list — an ordered sequence of tweet IDs, prepended on every fan-out write. The list is capped at a fixed length (e.g. 800 entries) by trimming older entries after each insert. Reads call LRANGE key 0 N-1 to get tweet IDs, then hydrate them from a separate tweet object store in one batch fetch.

Timeline cache structure

# Redis list per user, key: timeline:{user_id}
# Most-recent tweet ID is always at index 0 (head of list)
LPUSH timeline:usr_99  tweet_new           # prepend on fan-out
LTRIM timeline:usr_99  0 799               # cap at 800 entries

LRANGE timeline:usr_99 0 19                # read page 1 (first 20 IDs)

# Structure after several fan-out writes:
# index 0 → tweet_new  (most recent)
# index 1 → tweet_abc
# index 2 → tweet_xyz
# ...       (up to index 799, then trimmed)

Write-amplification worked example

A celebrity with 1 million followers posts a tweet. Under pure fan-out on write, here is exactly what happens — and why it cannot be done synchronously:

Even distributing fan-out across 100 worker threads, each handling 10 000 followers, that is still 100 000 Redis writes per thread — a sustained write storm that monopolises the cache tier. The celebrity exemption converts this from 1 M writes to 1 write (into the celebrity's own post list), deferring the merge to read time.

Hybrid read-time merge — concrete trace

User usr_99 follows 300 normal accounts and 4 celebrities (each > 10 000 followers). A GET /v1/timeline request executes these steps in parallel:

# Step 1: fetch precomputed timeline (normal fan-out accounts only)
LRANGE timeline:usr_99 0 49          # returns up to 50 tweet IDs, ~2 ms

# Step 2: fetch recent posts from each celebrity (parallel, not sequential)
LRANGE posts:celeb_a   0 49          # celebrity A's own post list
LRANGE posts:celeb_b   0 49
LRANGE posts:celeb_c   0 49
LRANGE posts:celeb_d   0 49          # total ~20 ms for 4 parallel fetches

# Step 3: hydrate tweet IDs → full tweet objects (batch GET from object store)
# ~5 ms

# Step 4: merge + sort by created_at descending, return top 20
# merge of ≤250 items in memory: <0.1 ms

# Total: ~27 ms → well within 100 ms p99 budget

Operating & debugging it

In production, two metrics tell you everything about the health of the timeline system: cache hit rate and fan-out queue lag. A cache hit rate drop below 99 % means readers are falling through to expensive DB reconstruction. Queue lag above ~5 seconds means recent posts aren't appearing in timelines — users notice this as "tweets missing." Both problems look like slow timelines but have completely different root causes.

To inspect the fan-out queue lag (assuming a Kafka-based fan-out worker):

1. Check cache hit rate first — a rate below 99 % almost always explains slow timelines before you investigate anything else.
2. Check fan-out queue lag — a growing lag means fresh tweets are not reaching follower caches.
3. Verify celebrity classification — use LLEN posts:celeb_X to confirm the post list exists and the timeline service is querying it.
4. For memory growth, audit key TTLs: every timeline key should have a TTL (24 h); keys without TTLs from inactive users accumulate indefinitely.
5. For duplicate tweets, trace a tweet ID through the fan-out worker log to confirm the idempotency check is running.

Sources & further reading

• InfoQ — Twitter engineering: timeline architecture and scalability
• Twitter/X API reference — official endpoint documentation
• Redis — Lists data type (the structure used for precomputed timeline caches)
• Twitter Engineering Blog (archive) — original fan-out architecture posts