Caching — Beyond "Just Add Redis".

Everyone knows: "Just add Redis, it'll be fast!"

Reality:

"There are only two hard things in Computer Science: cache invalidation and naming things."

— Phil Karlton

What makes caching hard

• When do you invalidate? Too early = wasted cache. Too late = stale data.
• What do you cache? Everything = memory explosion. Nothing = no benefit.
• How do you handle cache miss storms? 1 miss = 1 DB hit. 10K misses = DB crash.
• How do you stay consistent? User updates profile, sees old data. Confused.

Week 4 teaches you to cache strategically, not reflexively.

INVALIDATION STRATEGIES

1. TIME-TO-LIVE (TTL) — cache.set("product:123", data, ttl=300) # 5 minutes

• ok — Simple to implement
• ok — Self-healing (bad data expires)
• no — Stale for up to TTL duration
• no — All items expire at same rate (inventory vs descriptions)

2. EVENT-DRIVEN — When product updated → Publish event → Invalidate cache

• ok — Near real-time consistency
• ok — Only invalidate what changed
• no — Requires event infrastructure
• no — What if event is lost?

3. VERSIONED KEYS — cache.set(f"product:123:v{version}", data)

• ok — Old and new can coexist
• ok — Atomic switchover
• no — More complex key management
• no — Storage overhead

4. HYBRID (Reality) — Event-driven + short TTL as safety net. Best of both worlds.

• Normal operation: Cache has homepage data · 50,000 requests/second served from cache · Database: relaxing, 0 load
• Cache expires: All 50,000 requests simultaneously hit database · Database: "I'm in danger" · Response time: 10ms → 10,000ms → timeout · Users: "Site is down!"

This is the THUNDERING HERD.

SOLUTIONS

1. MUTEX/LOCKING — First request: Acquires lock, fetches from DB, updates cache. Other requests: Wait for lock, then read from cache. Problem: All requests waiting → high latency

2. PROBABILISTIC EARLY EXPIRATION — Don't expire at exact time. Random chance of refresh before expiration. Some requests refresh early, preventing stampede.

remaining = expires_at - now()          # seconds until expiry
if random() < 1.0 / max(remaining, 1):  # more likely as expiry nears
    refresh_cache()

3. REQUEST COALESCING — Multiple requests for same key → single DB query. All waiters get the same result.

async def get_with_coalesce(key):
    if key in pending_requests:
        return await pending_requests[key]

    # A Task (unlike a bare coroutine) can be awaited by many callers
    task = asyncio.create_task(fetch_from_db(key))
    pending_requests[key] = task
    try:
        return await task
    finally:
        del pending_requests[key]

4. BACKGROUND REFRESH — Never let cache expire. Background job refreshes before TTL. Cache always warm.

10 million users. Each has personalized feed. User posts → appears in all followers' feeds.

THE CELEBRITY PROBLEM

A celebrity posts → 300 million followers → Push to 300 million feed caches? → That's 300 million writes for 1 post!

SOLUTION: HYBRID

• Regular users (< 10K followers): Push on write
• Celebrities (> 10K followers): Pull on read

When building feed:

1. Fetch pre-computed feed (pushed posts)
2. Merge with celebrity posts (pulled on demand)
3. Return combined feed

Challenge

• 1 million products
• 100,000 requests/second
• Prices change daily
• Inventory changes every second
• Users must see accurate stock

Key decisions

• Different TTLs for different data?
• Event-driven for inventory?
• How to handle flash sales?

Challenge

• 50,000 requests/second
• Personalized for some users
• Changes hourly (featured products)
• Cache miss = database overwhelmed

Key decisions

• How to prevent thundering herd?
• Cache per user or shared?
• Background refresh strategy?

Challenge

• 10 million users
• Each follows 0-1000 accounts
• Some accounts have millions of followers
• Feed must feel "real-time"

Key decisions

• Push vs pull vs hybrid?
• How to handle celebrities?
• Cache per user or compute on read?

Challenge

• Mobile + Web clients
• Authenticated + Anonymous users
• CDN in front
• Multiple data types with different freshness needs

Key decisions

• What to cache at CDN vs App?
• How to handle authenticated caching?
• Cache invalidation across all tiers?

EXAMPLE: Product Update Flow

1. Admin updates product price (Week 2: Idempotency)
2. Update written to DB + Outbox (Week 3: Transactional Outbox)
3. Event published to Kafka (Week 3: Queue vs Stream)
4. Cache Invalidation Consumer receives event (Week 3: Consumer Groups)
5. Cache entry invalidated or updated (Week 4: Invalidation)
6. CDN cache purged via API (Week 4: Multi-tier)
7. Next request gets fresh data (Week 4: Cache-aside)

It's 12:00:00 AM on Black Friday. Your homepage cache TTL is 60 seconds. You have 100,000 users waiting to refresh.

• 12:00:00 — the cache expires.
• 12:00:01 — 100,000 requests hit your database.
• 12:00:02 — your database is on fire.

How do you prevent this? Day 3 will teach you:

1. Probabilistic early expiration
2. Request coalescing
3. Background refresh
4. Mutex with timeout
5. Cache warming strategies

You'll implement each one and know when to use which.