~learn.
?

Boss37 minIntro+60 XP

Design a Global E-Commerce Product Catalog System.

A Real-World Problem Covering Everything You've Learned in Week 4

6 phases6 code

Boss · W04 · 5 phases · 45 min

Fight it like the real interview: each phase runs against its time budget, you write your answer in a scratchpad first, then reveal the model answer side by side and grade yourself.

or read the model answer below
watch it first · newThe whole lesson in a 21-minute animated lecture5 quiz checkpoints · ask the AI about any moment · captions, chapters and speed

Intro · 1 min

The Interview Begins#

You walk into the interview room. The interviewer smiles and gestures to the whiteboard.

Interviewer

"Thanks for coming in. Today we're going to work through a system design problem together. I'm interested in your thought process, so please think out loud. Feel free to ask questions — this is meant to be collaborative."

They write on the whiteboard:

Design a Global E-Commerce Product Catalog System

You're building the product catalog for a large e-commerce platform like Amazon or Shopify. The system serves product pages, search results, category listings, and personalized recommendations.

Key challenges
  • 50 million products across 10,000 categories
  • 500 million daily page views globally
  • Flash sales with 100x traffic spikes
  • Price/inventory updates must reflect within seconds
  • Personalized "Recently Viewed" and "Recommended For You"

Focus on the CACHING ARCHITECTURE

Interviewer

"Take a few minutes to think about this, then walk me through your approach. We have about 45 minutes. I'm particularly interested in how you'd design the caching layers."

ExplainerThe whole capstone: a global e-commerce product catalog

50 million products, 500 million page views a day and flash sales that spike 100x: about 6,000 requests a second normally and 600,000 at peak, so a 1% miss rate would mean 6,000 database queries a second and the hit ratio must stay above 99%, on about 300 GB of Redis. Pages fall through browser, CDN, gateway and Redis, with prices on a separate 30-second, event-invalidated path. The deep dives protect the Deal of the Day from a thundering herd, invalidate each data type differently, tier personalization by activity and set a cache matrix where checkout is never cached; at 10x the database breaks first, and a Redis failure degrades to local caches.

Phase 1 · Stage 1 of 6 · 5 min budget · 2 min read

Phase 1: Requirements Clarification (5 minutes)#

Before diving in, you take a breath and start asking questions. This is crucial — never assume.

Your Questions#

Transcript
You

"Before I start designing, I'd like to clarify a few requirements. First, what's the read-to-write ratio? How often do products change?"

Interviewer

"Reads vastly outnumber writes. Product metadata like descriptions change maybe once a week. But prices can change multiple times per day, and inventory changes constantly — every purchase affects it."

You

"Interesting. So we have different freshness requirements for different data. For the flash sales — what's the traffic pattern? How quickly does it spike?"

Interviewer

"Flash sales are announced in advance but start at a specific time. Traffic goes from normal to 100x in about 30 seconds. We had an incident last Black Friday where the site went down at midnight."

You

"That sounds like a thundering herd problem. For personalization — the 'Recently Viewed' and recommendations — are these per-user or can they be shared?"

Interviewer

"Per-user. Each user has their own browsing history and recommendations based on their behavior."

You

"Got it. How global is the user base? Do we need multi-region deployment?"

Interviewer

"Yes, we have users in North America, Europe, and Asia. Latency matters — we want sub-200ms page loads globally."

You

"For the price changes, what's the acceptable staleness? Can a user see an old price for a few seconds?"

Interviewer

"For display purposes, a few seconds is acceptable. But at checkout, they MUST see the current price. We've had complaints about price mismatches."

You

"Last question — are there any hot products that get disproportionate traffic? Like featured items or viral products?"

Interviewer

"Absolutely. Our 'Deal of the Day' gets about 10% of all traffic. And when influencers mention products, those can spike to millions of views in minutes."

You

"Perfect. Let me summarize the requirements as I understand them."

Functional Requirements#

Functional requirements
1. PRODUCT DISPLAY
  • Product detail pages (title, description, images, specs)
  • Product pricing (current price, discounts, sale prices)
  • Product availability (inventory count, in-stock status)
  • Product reviews and ratings (aggregated)
2. PRODUCT DISCOVERY
  • Category browsing (hierarchical categories)
  • Search results (filtered, sorted, paginated)
  • Homepage featured products
3. PERSONALIZATION
  • Recently viewed products (per user)
  • Personalized recommendations (per user)
  • "Customers also bought" (per product)
4. REAL-TIME UPDATES
  • Flash sale price activation
  • Inventory updates on purchase
  • New product launches

Non-Functional Requirements#

Non-functional requirements
1. SCALE
  • 50 million products
  • 500 million page views/day (~6,000 req/sec average)
  • 100x spike during flash sales (~600,000 req/sec peak)
  • 100 million registered users
2. LATENCY
  • Product page load: <200ms p99 (globally)
  • Search results: <300ms p99
  • Personalization: <100ms p99
3. FRESHNESS (Bounded Staleness)
  • Product metadata: 1 hour acceptable
  • Price: <5 seconds during normal, immediate during checkout
  • Inventory: <30 seconds for display, real-time for purchase
4. AVAILABILITY
  • 99.9% uptime
  • Graceful degradation during failures
  • No downtime during flash sales

Phase 2 · Stage 2 of 6 · 5 min budget · 2 min read

Phase 2: Back-of-the-Envelope Estimation (5 minutes)#

Your turn · checkpoint 1not answered

Before the numbers: what hit ratio do you need?

The catalog serves 500 million page views a day, and flash sales bring 100x spikes. Roughly what are the average and peak request rates? At the peak, how many queries a second reach the database for every 1% of requests that miss the cache?

Think it through, or jot it down. Then compare.

One strong answer

  • Average: 500 million ÷ 86,400 s ≈ 6,000 requests/s.
  • Peak: 100 × 6,000 ≈ 600,000 requests/s.
  • Each 1% of misses at peak = 6,000 database queries/s, more than the database can take. So the cache must hit more than 99% of the time to survive a flash sale.
You

"Let me work through the numbers to understand the scale."

Traffic Estimation#

WorksheetPAGE VIEW TRAFFIC
  • Daily page views: 500 million
  • Seconds per day: 86,400
  • Average requests/sec: ~6,000 req/sec
  • Peak traffic (flash sale): 100x normal
  • Product detail pages: 60% → 3,600 req/sec (360K peak)
  • Category/search: 25% → 1,500 req/sec (150K peak)
  • Homepage: 10% → 600 req/sec (60K peak)
  • Personalization: 5% → 300 req/sec (30K peak)
  • Peak requests/sec: ~600,000 req/sec

Storage Estimation#

WorksheetStorage estimation
  • PRODUCT DATA — Products: 50 million
  • Metadata (title, desc): 5 KB
  • Pricing data: 100 bytes
  • Inventory data: 100 bytes
  • Images (URLs only): 500 bytes
  • Reviews aggregate: 200 bytes
  • Total per product: ~6 KB
  • Total product data: 50M × 6KB = 300 GB
  • PERSONALIZATION DATA — Users: 100 million
  • Recently viewed (20 items): 20 × 50 bytes = 1 KB per user
  • Recommendations (50 items): 50 × 50 bytes = 2.5 KB per user
  • Total per user: ~3.5 KB
  • Total personalization: 100M × 3.5KB = 350 GB

Cache Sizing#

WorksheetCACHE REQUIREMENTS
  • Product cache (hot products) — assume 20% products are "hot" (viewed daily): hot products 10 million, size 10M × 6KB = 60 GB, add overhead = ~80 GB Redis
  • Personalization cache — active users (daily) ~50 million, size 50M × 3.5KB = 175 GB, add overhead = ~200 GB Redis
  • Category/Search results cache — 10,000 categories × 50 variations = 500K entries, size 500K × 10KB = 5 GB, add overhead = ~10 GB Redis
  • TOTAL REDIS CLUSTER (distributed across multiple nodes): ~300 GB

Key Metrics Summary#

ESTIMATION SUMMARY
TRAFFIC
  • Average: 6,000 req/sec
  • Peak (flash sale): 600,000 req/sec
  • Target cache hit: >99% (to survive peak)
STORAGE
  • Product data: 300 GB (database)
  • Personalization: 350 GB (database)
  • Total cache: ~300 GB (Redis cluster)
INFRASTRUCTURE
  • Redis nodes: 10 nodes × 32 GB
  • CDN edge locations: 50+ global PoPs
  • API servers: 100 instances (auto-scaling)
CRITICAL INSIGHT

At 600K req/sec peak, even 1% cache miss = 6,000 DB queries/sec. Database cannot handle this. Cache hit rate must be >99%.

Quick check2 questions · +5 XP each

Estimating the catalog's load

  1. Q1/2 At the 600,000 req/s flash-sale peak, the cache hit ratio slips from 99% to 98%. What happens to database load?

    Answer & explanation

    Answer: A

    Database load tracks the miss ratio: 1% of 600K is 6,000, and 2% is 12,000. One point of hit ratio is a 2x swing in misses. That's why the estimate says the hit ratio must stay above 99% to survive the peak.

  2. Q2/2 The personalization cache is sized for ~50 million daily-active users, not all 100 million registered users. What's the reasoning?

    Answer & explanation

    Answer: C

    Cache size should follow the working set, not the total dataset. Users who aren't visiting don't generate hits, so their entries would just sit in memory. Their data still lives in the database and can be computed on demand if they return.

Phase 3 · Stage 3 of 6 · 10 min budget · 3 min read

Phase 3: High-Level Design (10 minutes)#

You

"Now let me sketch out the high-level architecture, focusing on the caching layers."

System Architecture#

DiagramMulti-tier caching architecture
%% title: Multi-tier caching architecture
flowchart TD
  C@{ icon: "sd:users", label: "Clients<br/>Browser / Mobile App / Third-party<br/>[Browser Cache Layer]" }
  CDN@{ icon: "tech:cloudfront", label: "CDN layer<br/>CloudFront / Fastly<br/>(50+ global edge locations)" }
  GW@{ icon: "sd:gateway", label: "API Gateway<br/>(Rate Limiting, Auth)" }
  PS["Product Service"]:::core
  SS["Search Service"]:::service
  PZ["Personalization"]:::service
  R@{ icon: "tech:redis", label: "Redis cluster<br/>Application Cache · 300GB across 10 nodes" }
  PG@{ icon: "tech:postgresql", label: "PostgreSQL<br/>(Products)" }
  ES@{ icon: "tech:elasticsearch", label: "Elasticsearch<br/>(Search)" }
  DY@{ icon: "tech:dynamodb", label: "DynamoDB<br/>(User Data)" }
  K@{ icon: "tech:kafka", label: "Event stream (Kafka)<br/>price.updated · inventory.changed · product.modified" }

  C --> CDN --> GW
  GW --> PS
  GW --> SS
  GW --> PZ
  PS --> R
  SS --> R
  PZ --> R
  R --> PG
  R --> ES
  R --> DY
  K -.->|"triggers cache invalidation"| R

  subgraph DBS["Databases"]
    direction TB
    PG
    ES
    DY
  end
  class C client
  class CDN,GW edge
  class R cache
  class PG,ES,DY store
  class K queue

CDN layer caches:

  • Static assets (JS, CSS, fingerprinted) - 1 year TTL; product images - 7 days TTL, purge on update
  • Product pages (anonymous; price and stock load via a separate uncached API call) - 60s TTL
  • Category pages - 60s TTL
  • NOT cached: personalized content, prices

Redis cluster keys:

  • Product Cache: product:{id} → full product data
  • Price Cache: price:{id} → current price
  • Inventory: inventory:{id} → stock count
  • Category Cache: category:{id}:page:{n} → products
  • User Feed: user:{id}:recent → product IDs
  • Recommendations: user:{id}:recs → product IDs

Component Breakdown#

You

"Let me walk through each component and its caching role..."

1. Browser Cache Layer#

Purpose: Cache static assets and reduce redundant requests

Strategy:

  • Static assets: Cache-Control: public, max-age=31536000, immutable
  • Anonymous product pages: Cache-Control: public, max-age=60, stale-while-revalidate=30 (so the CDN can cache them too)
  • Logged-in product pages: Cache-Control: private, max-age=0, must-revalidate
  • Personalized: Cache-Control: private, no-cache with ETag

2. CDN Layer#

Purpose: Serve content from edge locations globally for low latency

Strategy:

  • Product images: Long TTL (7 days), purge on update
  • Anonymous product pages: Short TTL (60s)
  • Search results: Short TTL (30s); the query string is part of the cache key
  • Personalized: Bypass CDN entirely

3. Application Cache (Redis)#

Purpose: Fast access to frequently requested data

Strategy:

  • Cache-aside pattern for most data
  • Event-driven invalidation + short safety TTL for critical data (prices, inventory)
  • Write-through for the user's own "recently viewed" list
  • Different TTLs based on data type

4. Event-Driven Invalidation#

Purpose: Keep caches fresh when data changes

Strategy:

  • Kafka events trigger invalidation
  • Invalidate app cache → gateway → CDN (in order)
  • Different strategies for different data types

Data Flow: Product Page Request#

You

"Let me trace through a typical product page request..."

PRODUCT PAGE REQUEST FLOW
  1. BROWSER CHECK — User requests: /products/ABC123
    • If cached and fresh → Return immediately
    • If stale → Send conditional request (If-None-Match)
  2. CDN CHECK
    • If HIT → Return from edge (20ms latency)
    • If MISS → Forward to origin
  3. API GATEWAY
    • Auth check (cached token validation)
    • Rate limit check
    • Route to Product Service
  4. PRODUCT SERVICE
    • Check Redis: product:ABC123
    • If HIT → Return cached product (2ms)
    • If MISS → Query PostgreSQL, cache result
  5. PRICE SERVICE (separate call)
    • Check Redis: price:ABC123
    • Short TTL (30s) + event-driven invalidation
    • Always fresh for checkout
  6. ASSEMBLE RESPONSE
    • Combine product + price + inventory
    • Set appropriate cache headers
    • Return to user

Total latency (cache hit): <50ms · Total latency (cache miss): <200ms

Quick check2 questions · +5 XP each

High-level caching design

  1. Q1/2 The design caches anonymous product pages at the CDN for 60 seconds but keeps prices out of the CDN. Why?

    Answer & explanation

    Answer: D

    Each tier's TTL has to fit the data's staleness budget. Descriptions tolerate an hour; prices allow only seconds, so they're served from Redis with event-driven invalidation instead. Prices in this system are the same for every user; the issue is freshness, not personalization.

  2. Q2/2 A shopper sees a product-page price that's a few seconds old. What in the design stops them from being charged a different price than the current one?

    Answer & explanation

    Answer: B

    Bounded staleness is fine for display, but the requirement is that checkout MUST use the current price, so that path bypasses the cache. TTLs and revalidation narrow the window but never guarantee the value is current at the moment of purchase.

Phase 4 · Stage 4 of 6 · 20 min budget · 22 min read

Phase 4: Deep Dives (20 minutes)#

Interviewer

"Great high-level design. Let's dive deeper into some specific challenges. How would you handle the flash sale traffic spike?"

Your turn · checkpoint 2not answered

Midnight, again

Last Black Friday the site went down at midnight. At 00:00 the "Deal of the Day" entry expires and 100,000 users refresh at once. What happens to the database, and what would you have in place before 00:00 so it can't happen again?

Think it through, or jot it down. Then compare.

One strong answer

  • Every request misses at once: about 100,000 database queries in a second against a database sized for normal traffic, so it falls over.
  • Before the sale: flash sales are scheduled, so warm the sale products a few minutes early and mark them hot for background refresh.
  • During the sale: serve stale data while refreshing in the background, coalesce concurrent misses into one fetch, and add jitter to TTLs so keys warmed together don't expire together.

Deep Dive 1: Thundering Herd Protection (Week 4, Day 3)#

You

"Flash sales are a classic thundering herd scenario. At midnight, thousands of users refresh simultaneously. If the cache expires or is empty, all those requests hit the database at once."

The Problem#

FLASH SALE THUNDERING HERD
  • 11:59:59 PM — Normal traffic —
    6,000 req/sec · 99% cache hit · 60 DB queries/sec
  • 12:00:00 AM — Flash sale starts —
    Cache expires on "Deal of the Day" product · 100,000 users refresh simultaneously · ALL requests miss cache · 100,000 DB queries in 1 second! · Database overwhelmed · Site goes down
  • Last Black Friday —
    This is what happened.

The Solution#

You

"I'd implement multiple layers of thundering herd protection..."

pythonCacheEntry · 290 lines
# Thundering Herd Protection Implementation
# Applies: Week 4, Day 3

import asyncio
import json
from dataclasses import dataclass
from typing import Optional, Dict, Any, Callable
from datetime import datetime, timedelta, timezone
import random
import logging

logger = logging.getLogger(__name__)


@dataclass
class CacheEntry:
    """Cache entry with fresh and stale TTLs."""
    value: Any
    fresh_until: datetime
    stale_until: datetime
    version: int = 0


class ThunderingHerdProtectedCache:
    """
    Cache with multiple thundering herd protections.
    
    Protections applied:
    1. Request coalescing - Duplicate requests share one fetch
    2. Stale-while-revalidate - Serve stale, refresh async
    3. TTL jitter - Stagger expiration times
    4. Background refresh - Pre-refresh hot keys before expiry
    """
    
    def __init__(self, redis_client, config: dict = None):
        self.redis = redis_client
        self.config = config or {}
        
        # In-flight requests for coalescing
        self._in_flight: Dict[str, asyncio.Future] = {}
        
        # Hot keys for background refresh
        self._hot_keys: set = set()
        
        # Metrics
        self._coalesced_count = 0
        self._stale_served_count = 0
    
    async def get(
        self,
        key: str,
        fetch_func: Callable,
        fresh_ttl: int = 60,
        stale_ttl: int = 300
    ) -> Any:
        """
        Get value with thundering herd protection.
        
        1. Check cache - if fresh, return immediately
        2. If stale but valid, return stale and refresh async
        3. If expired, coalesce requests and fetch once
        """
        # Try to get from cache
        entry = await self._get_entry(key)
        
        now = datetime.now(timezone.utc)
        
        # Case 1: Fresh cache hit
        if entry and entry.fresh_until > now:
            return entry.value
        
        # Case 2: Stale but within stale window - serve stale, refresh async
        if entry and entry.stale_until > now:
            self._stale_served_count += 1
            # Trigger async refresh (don't wait)
            asyncio.create_task(
                self._refresh_async(key, fetch_func, fresh_ttl, stale_ttl)
            )
            return entry.value
        
        # Case 3: Expired or missing - fetch with coalescing
        return await self._fetch_with_coalescing(
            key, fetch_func, fresh_ttl, stale_ttl
        )
    
    async def _fetch_with_coalescing(
        self,
        key: str,
        fetch_func: Callable,
        fresh_ttl: int,
        stale_ttl: int
    ) -> Any:
        """
        Fetch with request coalescing.
        
        If multiple requests come in for the same key,
        only one actually fetches - others wait for it.
        """
        # Check if there's already a fetch in flight
        if key in self._in_flight:
            self._coalesced_count += 1
            logger.debug(f"Coalescing request for {key}")
            return await self._in_flight[key]
        
        # Create future for this fetch
        future = asyncio.get_event_loop().create_future()
        self._in_flight[key] = future
        
        try:
            # Actually fetch the data
            value = await fetch_func()
            
            # Apply TTL jitter (random shortening of the fresh TTL)
            # to prevent synchronized expiration
            jitter = random.uniform(0.8, 1.0)
            actual_fresh_ttl = int(fresh_ttl * jitter)
            
            # Cache it
            await self._set_entry(key, value, actual_fresh_ttl, stale_ttl)
            
            # Complete the future
            future.set_result(value)
            return value
            
        except Exception as e:
            future.set_exception(e)
            raise
            
        finally:
            del self._in_flight[key]
    
    async def _refresh_async(
        self,
        key: str,
        fetch_func: Callable,
        fresh_ttl: int,
        stale_ttl: int
    ):
        """Background refresh without blocking."""
        try:
            # Don't refresh if already in flight
            if key in self._in_flight:
                return
            
            value = await fetch_func()
            
            jitter = random.uniform(0.8, 1.0)
            actual_fresh_ttl = int(fresh_ttl * jitter)
            
            await self._set_entry(key, value, actual_fresh_ttl, stale_ttl)
            logger.debug(f"Background refresh completed for {key}")
            
        except Exception as e:
            logger.warning(f"Background refresh failed for {key}: {e}")
    
    async def _get_entry(self, key: str) -> Optional[CacheEntry]:
        """Get cache entry with metadata."""
        data = await self.redis.hgetall(f"cache:{key}")
        if not data:
            return None
        
        return CacheEntry(
            value=json.loads(data[b'value']),
            fresh_until=datetime.fromisoformat(data[b'fresh_until'].decode()),
            stale_until=datetime.fromisoformat(data[b'stale_until'].decode()),
            version=int(data.get(b'version', 0))
        )
    
    async def _set_entry(
        self,
        key: str,
        value: Any,
        fresh_ttl: int,
        stale_ttl: int
    ):
        """Set cache entry with metadata."""
        now = datetime.now(timezone.utc)
        
        entry_data = {
            'value': json.dumps(value, default=str),
            'fresh_until': (now + timedelta(seconds=fresh_ttl)).isoformat(),
            'stale_until': (now + timedelta(seconds=fresh_ttl + stale_ttl)).isoformat(),
            'version': str(int(now.timestamp()))
        }
        
        pipe = self.redis.pipeline()
        pipe.hset(f"cache:{key}", mapping=entry_data)
        pipe.expire(f"cache:{key}", fresh_ttl + stale_ttl)
        await pipe.execute()
    
    # =========================================================================
    # Background Refresh for Hot Keys (Flash Sale Products)
    # =========================================================================
    
    def mark_hot(self, key: str):
        """Mark a key as hot for background refresh."""
        self._hot_keys.add(key)
    
    def unmark_hot(self, key: str):
        """Remove key from hot list."""
        self._hot_keys.discard(key)
    
    async def run_background_refresh(self, fetch_funcs: Dict[str, Callable]):
        """
        Background job to keep hot keys fresh.
        
        Run this continuously to ensure flash sale products
        are ALWAYS in cache before they're requested.
        """
        while True:
            for key in list(self._hot_keys):
                try:
                    entry = await self._get_entry(key)
                    
                    # Refresh if within 20% of fresh TTL expiring
                    if entry:
                        now = datetime.now(timezone.utc)
                        time_to_stale = (entry.fresh_until - now).total_seconds()
                        
                        if time_to_stale < 12:  # Less than 12 seconds to stale
                            if key in fetch_funcs:
                                await self._refresh_async(
                                    key, fetch_funcs[key], 60, 300
                                )
                
                except Exception as e:
                    logger.error(f"Background refresh error for {key}: {e}")
            
            await asyncio.sleep(5)  # Check every 5 seconds


# =============================================================================
# Flash Sale Cache Warming
# =============================================================================

class FlashSaleCacheWarmer:
    """
    Pre-warm cache before flash sales start.
    
    Flash sales are scheduled in advance, so we know
    which products will be hit. Warm them before midnight!
    """
    
    def __init__(self, cache: ThunderingHerdProtectedCache, db_client):
        self.cache = cache
        self.db = db_client
    
    async def warm_flash_sale(self, sale_id: str, start_time: datetime):
        """
        Warm cache for upcoming flash sale.
        
        Call this 5 minutes before sale starts.
        """
        # Get flash sale products
        products = await self.db.fetch(
            """
            SELECT p.* FROM products p
            JOIN flash_sale_items fsi ON p.id = fsi.product_id
            WHERE fsi.sale_id = $1
            """,
            sale_id
        )
        
        logger.info(f"Warming cache for {len(products)} flash sale products")
        
        for product in products:
            product_id = product['id']
            
            # Cache the product
            await self.cache._set_entry(
                f"product:{product_id}",
                dict(product),
                fresh_ttl=60,
                stale_ttl=300
            )
            
            # Mark as hot for background refresh
            self.cache.mark_hot(f"product:{product_id}")
        
        logger.info(f"Flash sale cache warming complete")
    
    async def cool_down_flash_sale(self, sale_id: str):
        """Remove products from hot list after sale ends."""
        products = await self.db.fetch(
            "SELECT product_id FROM flash_sale_items WHERE sale_id = $1",
            sale_id
        )
        
        for product in products:
            self.cache.unmark_hot(f"product:{product['product_id']}")

Edge Cases#

Transcript
Interviewer

"What if the background refresh fails?"

You

"If background refresh fails, the stale value is still served. Users see slightly old data (within stale window), but the system doesn't collapse. We'd alert on refresh failures and extend the stale TTL as a fallback."

Deep Dive 2: Multi-Strategy Cache Invalidation (Week 4, Day 2)#

Transcript
Interviewer

"You mentioned price changes need to reflect within seconds. How do you handle that across all cache layers?"

You

"This is a cache invalidation problem. Different data needs different strategies. Let me show how I'd handle it..."

The Problem#

INVALIDATION CHALLENGE
Product has multiple cached data types
Data Change rate Staleness
Description Changes weekly 1-hour staleness OK
Price Changes daily <5 second staleness required
Inventory Changes constantly <30 second staleness
Images Changes rarely 1-week staleness OK

One invalidation strategy doesn't fit all!

Additionally, data is cached at multiple tiers
  • Browser cache
  • CDN (50+ edge locations)
  • API Gateway
  • Redis (application cache)

Must invalidate in correct order!

The Solution#

pythonDataFreshness · 237 lines
# Multi-Strategy Cache Invalidation
# Applies: Week 4, Day 2 (Invalidation) + Day 5 (Multi-Tier)

from dataclasses import dataclass
from typing import List, Dict, Optional
from enum import Enum
import asyncio


class DataFreshness(Enum):
    """Data freshness requirements."""
    REAL_TIME = "real_time"      # <5 seconds, event-driven
    NEAR_TIME = "near_time"      # <30 seconds, event + short TTL
    EVENTUAL = "eventual"        # <1 hour, TTL only
    STATIC = "static"            # Days/weeks, TTL + purge on change


@dataclass
class InvalidationStrategy:
    """Strategy for a data type."""
    freshness: DataFreshness
    ttl: int
    event_driven: bool
    cdn_cache: bool
    cdn_ttl: int = 0


# Define strategies for different data types
INVALIDATION_STRATEGIES = {
    "product_metadata": InvalidationStrategy(
        freshness=DataFreshness.EVENTUAL,
        ttl=3600,           # 1 hour Redis TTL
        event_driven=False, # Just TTL-based
        cdn_cache=True,
        cdn_ttl=60          # 1 minute CDN
    ),
    "product_price": InvalidationStrategy(
        freshness=DataFreshness.REAL_TIME,
        ttl=30,             # 30 second Redis TTL (safety net for lost events;
                            # the <5 s target relies on events)
        event_driven=True,  # Event-driven invalidation
        cdn_cache=False     # Don't cache prices at CDN
    ),
    "product_inventory": InvalidationStrategy(
        freshness=DataFreshness.NEAR_TIME,
        ttl=30,             # 30 second Redis TTL
        event_driven=True,  # Event-driven invalidation
        cdn_cache=False     # Don't cache inventory at CDN
    ),
    "product_images": InvalidationStrategy(
        freshness=DataFreshness.STATIC,
        ttl=86400,          # 24 hour Redis TTL
        event_driven=True,  # Purge on image change
        cdn_cache=True,
        cdn_ttl=604800      # 7 day CDN
    ),
    "category_listing": InvalidationStrategy(
        freshness=DataFreshness.EVENTUAL,
        ttl=300,            # 5 minute Redis TTL
        event_driven=False, # Just TTL-based
        cdn_cache=True,
        cdn_ttl=60          # 1 minute CDN
    ),
}


class MultiStrategyInvalidationService:
    """
    Invalidation service with data-type-aware strategies.
    
    Different data types have different freshness requirements.
    This service applies the right strategy for each type.
    """
    
    def __init__(
        self,
        redis_client,
        cdn_client,
        event_consumer
    ):
        self.redis = redis_client
        self.cdn = cdn_client
        self.event_consumer = event_consumer
        self.strategies = INVALIDATION_STRATEGIES
    
    async def invalidate(
        self,
        data_type: str,
        entity_id: str,
        cascade: bool = True
    ):
        """
        Invalidate cache for an entity.
        
        Args:
            data_type: Type of data (determines strategy)
            entity_id: ID of the entity
            cascade: Whether to invalidate related caches
        """
        strategy = self.strategies.get(data_type)
        if not strategy:
            logger.warning(f"Unknown data type: {data_type}")
            return
        
        # Only process if event-driven
        if not strategy.event_driven:
            logger.debug(f"Skipping invalidation for TTL-only type: {data_type}")
            return
        
        # Invalidation order: App Cache → CDN
        # (Closest to database first)
        
        # 1. Application cache (Redis)
        await self._invalidate_redis(data_type, entity_id)
        
        # 2. CDN (if cached there)
        if strategy.cdn_cache:
            await self._invalidate_cdn(data_type, entity_id)
        
        # 3. Cascade to related caches if needed
        if cascade:
            await self._cascade_invalidation(data_type, entity_id)
    
    async def _invalidate_redis(self, data_type: str, entity_id: str):
        """Invalidate application cache."""
        keys = self._get_cache_keys(data_type, entity_id)
        
        if keys:
            await self.redis.delete(*keys)
            logger.info(f"Invalidated Redis keys: {keys}")
    
    async def _invalidate_cdn(self, data_type: str, entity_id: str):
        """Invalidate CDN cache."""
        urls = self._get_cdn_urls(data_type, entity_id)
        
        for url in urls:
            await self.cdn.purge_url(url)
        
        logger.info(f"Purged CDN URLs: {urls}")
    
    async def _cascade_invalidation(self, data_type: str, entity_id: str):
        """Invalidate related caches."""
        # Price change → invalidate product page cache
        if data_type == "product_price":
            await self._invalidate_redis("product_page", entity_id)
        
        # Inventory change → invalidate category listings
        if data_type == "product_inventory":
            categories = await self._get_product_categories(entity_id)
            for cat_id in categories:
                await self._invalidate_redis("category_listing", cat_id)
    
    def _get_cache_keys(self, data_type: str, entity_id: str) -> List[str]:
        """Get Redis keys for a data type and entity."""
        key_patterns = {
            "product_metadata": [f"product:{entity_id}:metadata"],
            "product_price": [f"product:{entity_id}:price", f"price:{entity_id}"],
            "product_inventory": [f"product:{entity_id}:inventory", f"inventory:{entity_id}"],
            "product_images": [f"product:{entity_id}:images"],
            "product_page": [f"product:{entity_id}", f"product_page:{entity_id}"],
            # NOTE: DEL does not expand globs; expand this pattern with SCAN, then UNLINK/DEL
            "category_listing": [f"category:{entity_id}:*"],
        }
        return key_patterns.get(data_type, [])
    
    def _get_cdn_urls(self, data_type: str, entity_id: str) -> List[str]:
        """Get CDN URLs to purge."""
        url_patterns = {
            "product_metadata": [f"/api/products/{entity_id}"],
            "product_images": [f"/images/products/{entity_id}/*"],
            "category_listing": [f"/api/categories/{entity_id}/*"],
        }
        return url_patterns.get(data_type, [])
    
    async def _get_product_categories(self, product_id: str) -> List[str]:
        """Get categories a product belongs to."""
        # Would query database
        return []


# =============================================================================
# Event Handler for Real-Time Invalidation
# =============================================================================

class PriceChangeEventHandler:
    """
    Handle price change events for real-time cache invalidation.
    
    When a price changes, we must invalidate immediately.
    """
    
    def __init__(self, invalidation_service: MultiStrategyInvalidationService):
        self.invalidation = invalidation_service
    
    async def handle(self, event: dict):
        """Handle price change event from Kafka."""
        product_id = event['product_id']
        old_price = event['old_price']
        new_price = event['new_price']
        
        logger.info(
            f"Price change detected: product={product_id}, "
            f"{old_price} → {new_price}"
        )
        
        # Invalidate price cache across all tiers
        await self.invalidation.invalidate(
            data_type="product_price",
            entity_id=product_id,
            cascade=True
        )
        
        # If significant price drop (flash sale), warm the cache
        if new_price < old_price * 0.5:  # 50%+ discount
            logger.info(f"Flash sale price detected for {product_id}")
            # Mark as hot for background refresh


class InventoryChangeEventHandler:
    """Handle inventory changes."""
    
    def __init__(self, invalidation_service: MultiStrategyInvalidationService):
        self.invalidation = invalidation_service
    
    async def handle(self, event: dict):
        """Handle inventory change event."""
        product_id = event['product_id']
        new_quantity = event['new_quantity']
        
        # Only invalidate if stock status changed
        # (in stock → out of stock, or vice versa)
        if event.get('stock_status_changed', False):
            await self.invalidation.invalidate(
                data_type="product_inventory",
                entity_id=product_id,
                cascade=True
            )

The Safety Net Pattern#

You

"I always combine event-driven invalidation with TTL as a safety net. Events can be lost or delayed. TTL ensures eventual consistency even if events fail."

pythoncache_product_price · 20 lines
# Safety Net Pattern: Event-Driven + TTL

async def cache_product_price(product_id: str, price: dict):
    """
    Cache price with safety net TTL.
    
    - Primary invalidation: Event-driven (real-time)
    - Safety net: 30-second TTL (eventual consistency)
    
    Even if the invalidation event is lost, price will
    be refreshed within 30 seconds. That backstop is looser
    than the <5 s price target, which holds while events flow;
    a lost event is the rare failure case (checkout re-reads
    the price, so no one is charged a stale one).
    """
    await redis.setex(
        f"price:{product_id}",
        30,  # Safety net TTL
        json.dumps(price)
    )
ExplainerOne product change, three invalidation strategies

Price: price.updated for ABC123 deletes product:ABC123:price and price:ABC123 in Redis first (prices are never cached at the CDN), then cascades to product:ABC123 and product_page:ABC123; if the event is lost, the 30-second safety-net TTL expires the price anyway, and checkout always re-reads it. Inventory: a purchase from 57 to 56 invalidates nothing (57 may show for up to 30 s); only a flip to out of stock deletes the inventory keys and cascades to category:{id}:*. Description: product_metadata is TTL only, so invalidate() skips it and copies live up to 1 hour in Redis and 60 s at the CDN.

Quick check3 questions · +5 XP each

Flash-sale protection and invalidation

  1. Q1/3 The capstone's cache multiplies each fresh TTL by a random factor between 0.8 and 1.0 when storing an entry. What problem does that address?

    Answer & explanation

    Answer: C

    Flash-sale products are warmed together, so with identical TTLs they'd also expire together and trigger a stampede. Jitter spreads their refreshes out. It shortens TTLs slightly at random; it doesn't favor popular keys or bound memory.

  2. Q2/3 Inventory events only invalidate the cache when stock status flips (in stock to out of stock, or back). A purchase drops a count from 57 to 56. What does the product page show, and why is that acceptable?

    Answer & explanation

    Answer: A

    Skipping invalidation for small count changes avoids an event storm; the 30-second safety TTL keeps displayed counts within the "<30 seconds for display" requirement. Purchases never rely on the cached count, so a slightly stale number can't cause overselling.

  3. Q3/3 During a flash sale, the invalidation consumer falls minutes behind on Kafka. What is the worst-case staleness of a price read from its price:{id} key in Redis?

    Answer & explanation

    Answer: D

    The safety-net TTL bounds staleness independently of event delivery: once a price entry expires, the next read reloads from the database. It is looser than the 5-second display target, which holds only while events flow. Lagging events stop helping, but they can't make a TTL'd entry live longer.

Deep Dive 3: Personalized Feed Caching (Week 4, Day 4)#

Transcript
Interviewer

"What about the personalized features? Recently Viewed and Recommendations are per-user. How do you cache those?"

You

"This is similar to social media feed caching. The challenge is that we can't pre-compute feeds for 100 million users. We need a hybrid approach."

The Problem#

PERSONALIZATION CACHE CHALLENGE

100 million users. Each user has:

  • Recently Viewed: 20 products
  • Recommendations: 50 products
If we cache feeds for all users
  • 100M users × 3.5KB = 350 GB
  • Most users are inactive (wasted storage)
If we compute on demand
  • Each view: Query user history + ML model
  • 100ms+ latency
  • Won't scale at peak traffic

The Solution#

You

"I'd use a hybrid caching strategy with activity-based tiering..."

pythonUserActivityTier · 294 lines
# Personalized Feed Caching
# Applies: Week 4, Day 4

from dataclasses import dataclass
from typing import List, Optional, Dict
from datetime import datetime, timedelta, timezone
from enum import Enum


class UserActivityTier(Enum):
    """User activity tiers for caching strategy."""
    ACTIVE = "active"       # Logged in today - full cache
    RECENT = "recent"       # Logged in this week - partial cache
    DORMANT = "dormant"     # Not logged in for 7+ days - compute on demand


@dataclass
class UserFeedConfig:
    """Configuration for user feed caching."""
    recently_viewed_limit: int = 20
    recommendations_limit: int = 50
    active_user_ttl: int = 3600       # 1 hour for active users
    recent_user_ttl: int = 86400      # 24 hours for recent users
    dormant_threshold_days: int = 7


class PersonalizedFeedService:
    """
    Personalized feed service with activity-based caching.
    
    Strategy:
    - Active users (today): Full cache, pre-computed
    - Recent users (this week): Partial cache, refresh on access
    - Dormant users (7+ days): Compute on demand, cache briefly
    """
    
    def __init__(
        self,
        redis_client,
        db_client,
        recommendation_service,
        config: UserFeedConfig = None
    ):
        self.redis = redis_client
        self.db = db_client
        self.recs = recommendation_service
        self.config = config or UserFeedConfig()
    
    async def get_recently_viewed(
        self,
        user_id: str,
        limit: int = 20
    ) -> List[dict]:
        """
        Get user's recently viewed products.
        
        Recently viewed is user-specific but simple:
        - Just a list of product IDs with timestamps
        - Easy to maintain incrementally
        """
        cache_key = f"user:{user_id}:recently_viewed"
        
        # Get from sorted set (most recent first)
        product_ids = await self.redis.zrevrange(
            cache_key, 0, limit - 1
        )
        
        if product_ids:
            return await self._get_products_by_ids(product_ids)
        
        # Cache miss - compute from database
        return await self._compute_recently_viewed(user_id, limit)
    
    async def record_view(self, user_id: str, product_id: str):
        """
        Record a product view.
        
        This is write-through: Write to cache AND database.
        Cache is always up-to-date.
        """
        now = datetime.now(timezone.utc).timestamp()
        cache_key = f"user:{user_id}:recently_viewed"
        
        pipe = self.redis.pipeline()
        
        # Add to sorted set (score = timestamp)
        pipe.zadd(cache_key, {product_id: now})
        
        # Trim to limit (keep most recent 20)
        pipe.zremrangebyrank(cache_key, 0, -self.config.recently_viewed_limit - 1)
        
        # Set TTL based on user activity
        tier = await self._get_user_tier(user_id)
        ttl = self._get_ttl_for_tier(tier)
        pipe.expire(cache_key, ttl)
        
        await pipe.execute()
        
        # Also persist to database (async)
        await self._persist_view(user_id, product_id)
    
    async def get_recommendations(
        self,
        user_id: str,
        limit: int = 20
    ) -> List[dict]:
        """
        Get personalized recommendations.
        
        Recommendations are expensive to compute (ML model).
        Strategy varies by user activity tier.
        """
        tier = await self._get_user_tier(user_id)
        
        if tier == UserActivityTier.ACTIVE:
            # Active users: Check cache, compute if missing
            return await self._get_active_user_recs(user_id, limit)
        
        elif tier == UserActivityTier.RECENT:
            # Recent users: Check cache, compute and cache if missing
            return await self._get_recent_user_recs(user_id, limit)
        
        else:
            # Dormant users: Compute on demand, cache briefly
            return await self._get_dormant_user_recs(user_id, limit)
    
    async def _get_active_user_recs(
        self,
        user_id: str,
        limit: int
    ) -> List[dict]:
        """Get recommendations for active users (cached)."""
        cache_key = f"user:{user_id}:recommendations"
        
        # Check cache
        cached = await self.redis.get(cache_key)
        if cached:
            product_ids = json.loads(cached)[:limit]
            return await self._get_products_by_ids(product_ids)
        
        # Cache miss - should be rare for active users
        # (Background job should have pre-computed)
        return await self._compute_and_cache_recs(
            user_id, limit, self.config.active_user_ttl
        )
    
    async def _get_recent_user_recs(
        self,
        user_id: str,
        limit: int
    ) -> List[dict]:
        """Get recommendations for recent users."""
        cache_key = f"user:{user_id}:recommendations"
        
        cached = await self.redis.get(cache_key)
        if cached:
            product_ids = json.loads(cached)[:limit]
            return await self._get_products_by_ids(product_ids)
        
        # Cache miss - compute and cache
        return await self._compute_and_cache_recs(
            user_id, limit, self.config.recent_user_ttl
        )
    
    async def _get_dormant_user_recs(
        self,
        user_id: str,
        limit: int
    ) -> List[dict]:
        """Get recommendations for dormant users."""
        # Don't check cache for dormant users
        # Their data would be stale anyway
        
        # Compute fresh recommendations
        recs = await self.recs.compute_recommendations(user_id, limit)
        
        # Cache briefly (5 minutes) in case they browse around
        cache_key = f"user:{user_id}:recommendations"
        await self.redis.setex(
            cache_key,
            300,
            json.dumps([r['product_id'] for r in recs])
        )
        
        return recs
    
    async def _compute_and_cache_recs(
        self,
        user_id: str,
        limit: int,
        ttl: int
    ) -> List[dict]:
        """Compute recommendations and cache."""
        recs = await self.recs.compute_recommendations(user_id, limit)
        
        cache_key = f"user:{user_id}:recommendations"
        await self.redis.setex(
            cache_key,
            ttl,
            json.dumps([r['product_id'] for r in recs])
        )
        
        return recs
    
    async def _get_user_tier(self, user_id: str) -> UserActivityTier:
        """Determine user's activity tier."""
        cache_key = f"user:{user_id}:last_active"
        
        last_active = await self.redis.get(cache_key)
        if not last_active:
            return UserActivityTier.DORMANT
        
        last_active_dt = datetime.fromisoformat(last_active.decode())
        days_inactive = (datetime.now(timezone.utc) - last_active_dt).days
        
        if days_inactive == 0:
            return UserActivityTier.ACTIVE
        elif days_inactive < self.config.dormant_threshold_days:
            return UserActivityTier.RECENT
        else:
            return UserActivityTier.DORMANT
    
    def _get_ttl_for_tier(self, tier: UserActivityTier) -> int:
        """Get cache TTL based on user tier."""
        ttls = {
            UserActivityTier.ACTIVE: self.config.active_user_ttl,
            UserActivityTier.RECENT: self.config.recent_user_ttl,
            UserActivityTier.DORMANT: 300  # 5 minutes
        }
        return ttls.get(tier, 300)
    
    async def _get_products_by_ids(self, product_ids: List[str]) -> List[dict]:
        """Fetch products by IDs with caching."""
        # Would use the product cache from Deep Dive 1
        pass
    
    async def _compute_recently_viewed(
        self,
        user_id: str,
        limit: int
    ) -> List[dict]:
        """Compute recently viewed from database."""
        pass
    
    async def _persist_view(self, user_id: str, product_id: str):
        """Persist view to database (async)."""
        pass


# =============================================================================
# Background Job: Pre-Compute Active User Recommendations
# =============================================================================

class RecommendationPreComputer:
    """
    Background job to pre-compute recommendations for active users.
    
    Run periodically (e.g., every hour) to ensure active users
    always have cached recommendations.
    """
    
    def __init__(
        self,
        feed_service: PersonalizedFeedService,
        db_client
    ):
        self.feed = feed_service
        self.db = db_client
    
    async def run(self):
        """Pre-compute recommendations for active users."""
        # Get users active in last 24 hours
        active_users = await self.db.fetch(
            """
            SELECT user_id FROM user_sessions
            WHERE last_active > NOW() - INTERVAL '24 hours'
            """
        )
        
        logger.info(f"Pre-computing recs for {len(active_users)} active users")
        
        for user in active_users:
            try:
                await self.feed._compute_and_cache_recs(
                    user['user_id'],
                    limit=50,
                    ttl=3600
                )
            except Exception as e:
                logger.warning(
                    f"Failed to pre-compute recs for {user['user_id']}: {e}"
                )
        
        logger.info("Recommendation pre-computation complete")

Deep Dive 4: Multi-Tier Cache Architecture (Week 4, Day 5)#

Transcript
Interviewer

"You mentioned different cache headers for different content types. Walk me through exactly what gets cached where."

You

"Let me detail the complete multi-tier strategy..."

The Cache Matrix#

MULTI-TIER CACHE MATRIX
Content Type Browser CDN Gateway Redis
Static Assets (JS, CSS) 1 year, immutable 1 year, immutable N/A N/A
Product Images 1 week 1 week, purge on chg N/A N/A
Product Page (anonymous) 60s, swr=30s 60s, purge on chg 30s 5 min, event inv.
Product Page (authenticated) 0, private NO (private) NO (auth + rate limits only) 5 min, event inv.
Product Price 0, no-cache NO (dynamic) NO 30s, event inv.
Inventory 0, no-cache NO (dynamic) NO 30s, event inv.
Category Page (anonymous) 60s, swr=30s 60s 30s 5 min
Search Results 30s, query string in cache key 30s, query string in cache key NO 1 min
Recently Viewed (user-specific) 0, private NO NO 1 hour, write-thru
Recommendations (user-specific) 0, private NO NO varies by tier
Cart 0, no-store NO NO 30 min
Checkout 0, no-store NO NO NO (real-time)

Key: swr = stale-while-revalidate · event inv. = event-driven invalidation · write-thru = write-through caching

Implementation#

pythonContentType · 180 lines
# Multi-Tier Cache Headers
# Applies: Week 4, Day 5

from fastapi import FastAPI, Response, Request, Depends
from enum import Enum


class ContentType(Enum):
    STATIC_ASSET = "static_asset"
    PRODUCT_IMAGE = "product_image"
    PRODUCT_PAGE_ANON = "product_page_anon"
    PRODUCT_PAGE_AUTH = "product_page_auth"
    PRODUCT_PRICE = "product_price"
    INVENTORY = "inventory"
    CATEGORY_PAGE = "category_page"
    SEARCH_RESULTS = "search_results"
    PERSONALIZED = "personalized"
    CART = "cart"
    CHECKOUT = "checkout"


class CacheHeaderBuilder:
    """
    Build appropriate cache headers for each content type.
    
    Different content requires different caching strategies
    across browser, CDN, and gateway layers.
    """
    
    CACHE_POLICIES = {
        ContentType.STATIC_ASSET: {
            "cache_control": "public, max-age=31536000, immutable",
            "cdn_cache": True,
            "vary": ["Accept-Encoding"],
        },
        ContentType.PRODUCT_IMAGE: {
            "cache_control": "public, max-age=604800",  # 1 week
            "cdn_cache": True,
            "vary": ["Accept-Encoding"],
        },
        ContentType.PRODUCT_PAGE_ANON: {
            "cache_control": "public, max-age=60, stale-while-revalidate=30",
            "cdn_cache": True,
            "vary": ["Accept-Encoding", "Accept-Language"],
        },
        ContentType.PRODUCT_PAGE_AUTH: {
            "cache_control": "private, max-age=0, must-revalidate",
            "cdn_cache": False,
            "vary": ["Authorization", "Accept-Encoding"],
        },
        ContentType.PRODUCT_PRICE: {
            "cache_control": "no-cache, no-store, must-revalidate",
            "cdn_cache": False,
            "vary": [],
        },
        ContentType.INVENTORY: {
            "cache_control": "no-cache, no-store, must-revalidate",
            "cdn_cache": False,
            "vary": [],
        },
        ContentType.CATEGORY_PAGE: {
            "cache_control": "public, max-age=60, stale-while-revalidate=30",
            "cdn_cache": True,
            "vary": ["Accept-Encoding", "Accept-Language"],
        },
        ContentType.SEARCH_RESULTS: {
            "cache_control": "public, max-age=30",
            "cdn_cache": True,
            "vary": ["Accept-Encoding"],  # Also varies on query string
        },
        ContentType.PERSONALIZED: {
            "cache_control": "private, no-cache",
            "cdn_cache": False,
            "vary": ["Authorization"],
        },
        ContentType.CART: {
            "cache_control": "private, no-store",
            "cdn_cache": False,
            "vary": [],
        },
        ContentType.CHECKOUT: {
            "cache_control": "no-store",
            "cdn_cache": False,
            "vary": [],
        },
    }
    
    @classmethod
    def get_headers(
        cls,
        content_type: ContentType,
        etag: str = None
    ) -> dict:
        """Get cache headers for content type."""
        policy = cls.CACHE_POLICIES[content_type]
        
        headers = {
            "Cache-Control": policy["cache_control"],
        }
        
        if policy["vary"]:
            headers["Vary"] = ", ".join(policy["vary"])
        
        if etag:
            headers["ETag"] = f'"{etag}"'
        
        # Add CDN hints
        if not policy["cdn_cache"]:
            headers["CDN-Cache-Control"] = "no-store"
        
        return headers


# FastAPI Integration
app = FastAPI()


def get_auth_status(request: Request) -> bool:
    """Check if request is authenticated."""
    return "Authorization" in request.headers


@app.get("/api/products/{product_id}")
async def get_product(
    product_id: str,
    response: Response,
    authenticated: bool = Depends(get_auth_status)
):
    """Get product with appropriate cache headers."""
    
    # Fetch product (from cache or DB)
    product = await product_service.get_product(product_id)
    
    # Determine content type based on auth status
    content_type = (
        ContentType.PRODUCT_PAGE_AUTH if authenticated
        else ContentType.PRODUCT_PAGE_ANON
    )
    
    # Set cache headers
    headers = CacheHeaderBuilder.get_headers(
        content_type,
        etag=str(product['updated_at'])
    )
    
    for header, value in headers.items():
        response.headers[header] = value
    
    return product


@app.get("/api/products/{product_id}/price")
async def get_price(product_id: str, response: Response):
    """
    Get product price.
    
    Never cached at CDN - always fresh from Redis/DB.
    """
    price = await price_service.get_price(product_id)
    
    headers = CacheHeaderBuilder.get_headers(ContentType.PRODUCT_PRICE)
    for header, value in headers.items():
        response.headers[header] = value
    
    return price


@app.get("/api/me/recently-viewed")
async def get_recently_viewed(
    response: Response,
    user = Depends(get_current_user)
):
    """Get user's recently viewed - personalized, private."""
    items = await feed_service.get_recently_viewed(user.id)
    
    headers = CacheHeaderBuilder.get_headers(ContentType.PERSONALIZED)
    for header, value in headers.items():
        response.headers[header] = value
    
    return items
Quick check3 questions · +5 XP each

Personalization and the cache matrix

  1. Q1/3 Recommendations are pre-computed hourly for active users but computed on demand (and cached for 5 minutes) for dormant users. Why not pre-compute for everyone?

    Answer & explanation

    Answer: B

    This is the feed lesson's activity-based tiering: spend pre-computation where reads are likely, and pay the on-demand cost only for the rare dormant user who returns. The short 5-minute cache covers that user browsing a few pages without recomputing each time.

  2. Q2/3 Why does record_view write "recently viewed" to Redis and the database together (write-through), instead of cache-aside?

    Answer & explanation

    Answer: C

    With write-through, the user's own action is visible on their next request. With cache-aside, a cached list could keep showing the old history until it expires. It still writes to the database on each view; it just also updates the cache in the same step.

  3. Q3/3 Search results are cached at the CDN for 30 seconds with Vary: Accept-Encoding only. What keeps a cached ?q=shoes response from being served for ?q=laptops?

    Answer & explanation

    Answer: A

    By default a CDN keys on URL plus query string, so different q values are different entries. Vary adds header-based dimensions on top of that. Search results here are public and shared across users, which is what makes edge caching them worthwhile.

Phase 5 · Stage 5 of 6 · 5 min budget · 3 min read

Phase 5: Scaling and Edge Cases (5 minutes)#

Interviewer

"How would this system scale to 10x the current load? What breaks first?"

Your turn · checkpoint 3not answered

10x: what breaks first?

At 10x, the peak is 6 million requests a second and the cache grows past a terabyte. Rank the CDN, the Redis cluster and the database by how likely each is to break first, and say why.

Think it through, or jot it down. Then compare.

One strong answer

  • CDN: lowest risk. It scales out by design; you add edge capacity and pay roughly linearly.
  • Redis: medium. Add nodes (30+ larger nodes instead of 10) and keep sharding by product ID.
  • Database: highest. Every miss still lands on it, and even a 99% hit ratio leaves 1% of 6 million = 60,000 queries/s, beyond a single PostgreSQL. It needs read replicas, connection pooling or a store built for that read rate.

Scaling Strategy#

You

"Let me analyze the bottlenecks at 10x scale..."

SCALING ANALYSIS: 10X LOAD

Current → 10x:

  • 6K req/sec → 60K req/sec average
  • 600K req/sec → 6M req/sec peak
  • 300 GB cache → 1+ TB cache
BOTTLENECK ANALYSIS
Component Risk Notes
1. CDN Lowest risk CDN scales horizontally by design · More edge locations as needed · Cost scales linearly
2. Redis Cluster Medium risk Current: 10 nodes × 32 GB = 320 GB · 10x: 30+ nodes × 64 GB = 2 TB · Challenge: Cross-slot operations · Solution: Shard by product ID consistently
3. Database Highest risk Even with 99% cache hit, 1% miss at 6M req/sec = 60K DB queries/sec · PostgreSQL won't handle this · Solutions: Read replicas (10+), Connection pooling (PgBouncer), Consider DynamoDB for product reads
4. Network Medium risk Internal traffic between services · Solution: Co-locate services, use service mesh

Edge Cases#

Transcript
Interviewer

"What happens if Redis goes down during a flash sale?"

You

"That's our worst-case scenario. Here's how we handle it..."

EDGE CASE: REDIS CLUSTER FAILURE

Scenario: Redis cluster partially fails during flash sale · 600K req/sec hitting the system

Impact WITHOUT mitigation: All requests hit database → database fails → site down

MITIGATION STRATEGY
  1. CIRCUIT BREAKER — Detect Redis failure quickly (< 1 second) · Open circuit, stop trying Redis · Serve degraded experience
  2. GRACEFUL DEGRADATION — Serve stale data from local memory cache (Guava/Caffeine) · Show "temporarily unavailable" for personalization · Block flash sale purchases temporarily (prevent oversell)
  3. FALLBACK DATA — Pre-compute "default" product data · Store in local process memory · Serve default when cache unavailable
  4. AUTOMATIC RECOVERY — Circuit breaker half-open after 30 seconds · Test with single request · Gradually restore traffic if successful
pythonResilientProductService · 63 lines
# Graceful Degradation Implementation

class ResilientProductService:
    """
    Product service with graceful degradation.
    
    Falls back to local cache → default data → error
    if Redis is unavailable.
    """
    
    def __init__(self, redis_cache, local_cache, db_client):
        self.redis = redis_cache
        self.local = local_cache  # In-memory (Guava-style)
        self.db = db_client
        self.circuit_breaker = CircuitBreaker(
            failure_threshold=5,
            recovery_timeout=30
        )
    
    async def get_product(self, product_id: str) -> dict:
        """Get product with fallback chain."""
        
        # Try Redis (primary cache)
        if self.circuit_breaker.is_closed():
            try:
                product = await self.redis.get(f"product:{product_id}")
                if product:
                    # Also store in local cache for fallback
                    self.local.put(product_id, product)
                    return product
            except Exception as e:
                self.circuit_breaker.record_failure()
                logger.warning(f"Redis failure: {e}")
        
        # Try local in-memory cache (fallback)
        product = self.local.get(product_id)
        if product:
            logger.info(f"Serving from local cache: {product_id}")
            return product
        
        # Try database (last resort during outage)
        if self.circuit_breaker.is_open():
            # Don't hammer DB during Redis outage
            # Return default/error
            return self._get_default_product(product_id)
        
        # Normal cache miss - fetch from DB
        product = await self.db.fetch_product(product_id)
        if product:
            await self.redis.set(f"product:{product_id}", product)
            self.local.put(product_id, product)
        
        return product
    
    def _get_default_product(self, product_id: str) -> dict:
        """Return minimal product data during outage."""
        return {
            "id": product_id,
            "title": "Product Temporarily Unavailable",
            "price": None,
            "inventory": None,
            "_degraded": True
        }

Failure Scenarios#

Failure Detection Impact Recovery
Redis node failure Health checks Partial cache miss Auto-failover to replica
Redis cluster failure Circuit breaker High DB load Local cache + degradation
CDN outage Synthetic monitoring Higher origin load Bypass CDN, scale origin
Database slow Query latency alerts Slow responses Read replicas, cache more
Kafka consumer lag Lag monitoring Stale prices Alert, scale consumers
Quick check2 questions · +5 XP each

Scaling and failure handling

  1. Q1/2 At 10x load (6 million req/s peak), why does the analysis rank the database, not the CDN or Redis, as the highest risk?

    Answer & explanation

    Answer: D

    Caches and CDNs scale out by adding nodes, but every miss still lands on the database: 1% of 6M is 60K queries/s. That's why the mitigations are read replicas, connection pooling, or a more scalable store for product reads.

  2. Q2/2 Redis fails during a flash sale and the circuit breaker opens. When the local in-memory cache also misses, ResilientProductService returns a degraded default instead of querying the database. Why?

    Answer & explanation

    Answer: B

    With Redis gone, the database would get the whole peak load it was never sized for. Degrading a few product views keeps the rest of the site up. The breaker then goes half-open after its timeout and tests Redis before traffic is restored.

Phase 6 · Stage 6 of 6 · 2 min

Phase 6: Monitoring and Operations#

Interviewer

"How would you monitor this caching system in production?"

Key Metrics#

You

"I'd track metrics at multiple levels..."

Cache Metrics#

CACHE HEALTH DASHBOARD — CACHE PERFORMANCE
Area Metric Current Target / alert
REDIS CLUSTER Hit ratio 98.5% Target: >99%
Latency p99 2.1ms Target: <5ms
Memory usage 78% Alert: >85%
Connections 4,521 Max: 10K
Evictions/sec 12 Alert: >100
CDN Hit ratio 94.2% Target: >90%
Origin requests/sec 358
Bandwidth saved 89%
Purges pending 12
INVALIDATION Events processed/sec 245
Consumer lag 34 Alert: >1000
Failed invalidations 0 Alert: >0
THUNDERING HERD PROTECTION Coalesced requests 12,456 (requests saved)
Stale served 2,341 (graceful degradation)
Background refreshes 892 (proactive updates)

Alerting Strategy#

Alerting strategy

CRITICAL — PagerDuty, wake up

  • Cache hit ratio < 95% for 5 minutes
  • Redis latency p99 > 50ms for 5 minutes
  • Redis cluster node down
  • Invalidation consumer lag > 10,000
  • Circuit breaker opened

WARNING — Slack, business hours

  • Cache hit ratio < 98% for 15 minutes
  • Redis memory > 85%
  • CDN origin requests > 1000/sec
  • Eviction rate > 100/sec

INFO — Dashboard only

  • Cache key expiration patterns
  • Hot key detection
  • Invalidation event volume

Runbook: Cache Hit Ratio Drop#

RUNBOOK: Cache Hit Ratio Below 95%
SYMPTOMS
  • Alert: "Redis cache hit ratio dropped to X%"
  • Increased database latency
  • Increased API response times
DIAGNOSIS
  1. Check for recent deployments: kubectl get deployments -n production --sort-by='.metadata.creationTimestamp'
  2. Check cache key version (did we bump it accidentally?): redis-cli GET cache_version
  3. Check for mass invalidation events: kafka-consumer-groups.sh --bootstrap-server <broker:9092> --describe --group cache-invalidation
  4. Check hot keys (thundering herd?): redis-cli --hotkeys
  5. Check memory pressure (evictions?): redis-cli INFO stats | grep evicted
RESOLUTION
  1. If deployment issue: Rollback deployment · Investigate cache key changes
  2. If thundering herd: Enable emergency background refresh · Increase stale TTL temporarily
  3. If memory pressure: Scale Redis cluster (add nodes) · Review TTLs (reduce if possible)
  4. If invalidation storm: Pause non-critical invalidation · Investigate source of invalidations
ESCALATION
  • If not resolved in 15 minutes: Page on-call SRE
  • If database impacted: Page database team
Quick check3 questions · +5 XP each

Operating the cache

  1. Q1/3 The Redis hit ratio collapses minutes after a deploy. Per the runbook, what is the first thing to suspect?

    Answer & explanation

    Answer: B

    A key-format or version change makes the whole cache look empty at once. Check cache_version and roll back if the deploy caused it.

  2. Q2/3 Why does "failed invalidations > 0" alert, while up to 100 evictions a second is fine?

    Answer & explanation

    Answer: B

    Evictions hurt performance a little; a missed invalidation hurts correctness, showing customers the wrong price or stock until the TTL saves you.

  3. Q3/3 The dashboard counts 12,456 "coalesced requests". What did coalescing save?

    Answer & explanation

    Answer: B

    When many requests miss on the same hot key at once, one goes to the database and the rest wait for its result. That's the thundering-herd protection at work.

Wrap-up · 2 min

Interview Conclusion#

Transcript
Interviewer

"Excellent work. You've demonstrated strong understanding of caching patterns, clear trade-off decisions, and practical production experience. Any questions for me?"

You

"Thank you! I'd love to hear how your team currently handles cache invalidation for pricing updates. Do you use event-driven invalidation, and if so, what message broker do you use?"

Interviewer

"We actually use a combination — Kafka for inventory and pricing events, and simple TTL for product metadata. We've had some challenges with flash sales similar to what you described. Your thundering herd protection approach is something we should consider."

You

"That's great to hear. I'm also curious about your CDN setup — do you use a single provider or multi-CDN?"

Summary: Week 4 Concepts Applied#

Week 4 Concepts (Caching — Beyond "Just Add Redis")#

Day Concept Application in This Design
Day 1: Caching Patterns Cache-aside, write-through Cache-aside for products, write-through for recently viewed
Day 2: Invalidation Event-driven, TTL safety net, multi-strategy Different strategies per data type (price=event, metadata=TTL)
Day 3: Thundering Herd Request coalescing, stale-while-revalidate, background refresh Flash sale protection, hot key handling
Day 4: Feed Caching Activity-based tiering, hybrid push/pull Recently viewed (write-through), recommendations (tiered)
Day 5: Multi-Tier Browser, CDN, Gateway, App cache layers Complete cache matrix with appropriate headers per tier

Code Patterns Demonstrated#

Code patterns demonstrated
  • 1. THUNDERING HERD PROTECTION
    • Request coalescing with asyncio.Future
    • Stale-while-revalidate pattern
    • Background refresh for hot keys
    • Flash sale cache warming
  • 2. MULTI-STRATEGY INVALIDATION
    • Data-type-aware invalidation
    • Event-driven for real-time data
    • TTL safety net for all data
    • Cascade invalidation (price → product page)
  • 3. ACTIVITY-BASED PERSONALIZATION
    • User tier classification (active/recent/dormant)
    • Different TTLs per tier
    • Background pre-computation for active users
  • 4. MULTI-TIER CACHE HEADERS
    • Content-type-aware headers
    • CDN-Cache-Control for edge control
    • Vary headers for correct caching
    • Private vs public distinction
  • 5. GRACEFUL DEGRADATION
    • Circuit breaker pattern
    • Local cache fallback
    • Default data for outages

Self-Assessment Checklist#

After studying this capstone, you should be able to:

After studying this capstone, you should be able to0/11 ready
  • Design a multi-tier caching architecture (browser → CDN → gateway → app → DB)
  • Choose appropriate caching patterns for different data types
  • Implement thundering herd protection with request coalescing
  • Design event-driven cache invalidation with TTL safety nets
  • Apply different invalidation strategies based on data freshness requirements
  • Implement activity-based caching for personalized data
  • Set correct HTTP cache headers for anonymous vs authenticated content
  • Handle cache failures gracefully with degradation strategies
  • Design cache warming strategies for predictable traffic spikes
  • Monitor cache health and troubleshoot performance issues
  • Estimate cache sizes and hit ratio requirements for a given load

This capstone integrates all concepts from Week 4 of the System Design Mastery Series: Caching — Beyond "Just Add Redis". Use this as a template for approaching e-commerce and high-traffic system design interviews.

this lesson in shorts · 2

play them all →

One idea from this lesson in about a minute each, made for your phone.

End of Boss · Design a Global E-Commerce Product Catalog System
Finish BossStages clear as you read them (0/6 so far). Skipping ahead doesn't count: marking complete unlocks after ~7 more min of reading.

Go deeper · reading trail

all reading →

Real write-ups from the teams that built these systems. Each opens on the publisher's site in a new tab.

AmazonBlog post · ~15 min

Caching challenges and strategies

Jas Chhabra · 2019
What
Amazon's lessons on running caches in front of services.
Key idea
A fleet-wide cache flush or a cold start can overload the backend; request coalescing and careful TTLs keep the service stable.
Why here
Directly useful for the capstone's thundering-herd deep dive on a global catalog after deploys or flash sales.
taught here: Cold cache and request coalescing
Dream11Blog post · ~6 min

How does Dream11 serve millions of Personalised Contests in a match

2021
What
How Dream11 shows millions of users a personalised list of contests just before a match starts.
Key idea
Spark Streaming writes contests into Aerospike, and hot keys are seeded as several copies read by userId modulo copies, so no single node serves everyone.
Why here
Two capstone deep dives at once: personalised listings built from shared cached data, and hot-key spreading by key replication.
taught here: Personalized feed caching

New lessons by email

New lessons, side quests and case files as they ship, at most once a week. No spam, unsubscribe anytime.

Discussion · …

Public: everyone reading this lesson sees comments. Be kind and keep it on the material.

Sign in to ask a question or answer one. Reading stays open to everyone.
loading the discussion…