Fight it like the real interview: each phase runs against its time budget, you write your answer in a scratchpad first, then reveal the model answer side by side and grade yourself.
The Interview Begins#
You walk into the interview room. The interviewer smiles and gestures to the whiteboard.
"Thanks for coming in. Today we're going to work through a system design problem together. I'm interested in your thought process, so please think out loud. Feel free to ask questions — this is meant to be collaborative."
They write on the whiteboard:
TinyLink is a URL shortening service like bit.ly or tinyurl.com. Users paste long URLs and get short, shareable links.
- 100 million new URLs created per day
- 10:1 read to write ratio (1 billion redirects/day)
- URLs should be as short as possible
- Links expire after 5 years by default
- Analytics: track click counts per URL
- API access for enterprise customers
"Take a few minutes to think about this, then walk me through your approach. We have about 45 minutes."
TinyLink creates 100 million 7-character links a day with ten redirects per creation; the redirect must take under 100 ms at p99 with 99.99% uptime. That is about 1,160 creates and 11,600 redirects a second with a viral design target of 120,000, and 182.5 billion URLs (about 300 TB replicated) out of 3.5 trillion possible codes. Redirects go CDN, load balancer, Redirect Service, Redis, then sharded PostgreSQL, creation is a separate API service, and clicks go to Kafka; the deep dives allocate ranges of 10,000 counter values per server, absorb viral links at the CDN (about 80%) and in local caches, Redis replicas and suffixed keys, shard by a hash ring on short_code with a user_id index, rate-limit tiers from 100 to 100,000 requests a minute, and choose consistency per operation, then scale 10x by adding instances, nodes and shards.
Phase 1: Requirements Clarification (5 minutes)#
Before diving in, you take a breath and start asking questions. This is crucial — never assume.
Your Questions#
"Before I start designing, I'd like to clarify a few requirements. With 100 million URLs per day, are these evenly distributed, or do we have peak hours?"
"Good question. Traffic is global, so it's relatively even, with daily peaks around twice the average. But during major events — like when a viral tweet includes a TinyLink — we can see 10x spikes for specific URLs."
"For the short URLs — do they need to be truly random, or can they be sequential? And what characters can we use?"
"They should appear random to users for security reasons. Use alphanumeric characters — uppercase, lowercase, and digits. So 62 possible characters."
"What happens when a user creates a short URL for the same long URL twice? Same short URL or different?"
"Different. Each creation is unique. Users might share the same destination but want separate tracking."
"For the analytics — do users need real-time click counts, or is eventual consistency acceptable?"
"Eventually consistent is fine. A few seconds of delay is acceptable. But the redirect itself must be fast — that's the critical path."
"Last question — you mentioned enterprise API access. Do we need to rate limit API consumers?"
"Absolutely. Free tier gets 100 requests/minute, paid tiers get more. We've had issues with abuse in the past."
"Perfect. Let me summarize the requirements as I understand them."
Functional Requirements#
1. URL shortening
- Create short URL from long URL
- Short codes should be 7 characters (alphanumeric)
- Each creation generates unique short URL
- Support custom aliases (user-defined short codes)
2. URL redirection
- Redirect short URL to original long URL
- Must be fast (<100ms p99)
- Handle non-existent URLs gracefully
3. Analytics
- Track total clicks per URL
- Track clicks by day (for charts)
- Eventually consistent (seconds of delay OK)
4. URL management
- URLs expire after 5 years by default
- Users can set custom expiration
- Users can delete their URLs
- Users can view their created URLs
5. API access
- RESTful API for create/read/delete
- Authentication via API keys
- Rate limiting per tier (free/paid)
Non-Functional Requirements#
1. Scale
- 100M URL creations per day (~1,200/sec average)
- 1B redirects per day (~12,000/sec average)
- Peak traffic: 10x for viral URLs (120,000 redirects/sec)
- Storage: 5 years of URLs
2. Reliability
- No duplicate short codes (uniqueness guaranteed)
- No lost URLs (durability)
- Redirects must work even during partial failures
3. Latency
- URL creation: <500ms at p99
- Redirect: <100ms at p99
- Analytics read: <1s at p99
4. Availability
- 99.99% uptime for redirects (critical path)
- 99.9% uptime for creation (less critical)
Before the numbers: size TinyLink
100 million new URLs a day, 10 redirects for every URL created, viral spikes of 10×, links kept for 5 years, codes drawn from 62 characters. What redirect rate do you design for? How many codes do 5 years need, and are 6 characters enough, or 7?
One strong answer
- Redirects: 1 billion a day ÷ 86,400 ≈ 11,600 a second on average; at 10× that's ~120,000 a second, the design target.
- Codes: 100 million × 365 × 5 ≈ 182.5 billion.
- Length: 62^6 ≈ 57 billion is too few; 62^7 ≈ 3.5 trillion leaves the space ~5% used. 7 characters.
Phase 2: Back of the Envelope Estimation (5 minutes)#
"Let me work through the numbers to understand the scale we're dealing with."
Traffic Estimation#
- Daily creates: 100,000,000
- Seconds per day: 86,400
- Average writes/sec: 100M / 86,400 ≈ 1,160 /sec
- Peak write traffic (2x): ~2,500 /sec
- Daily redirects: 1,000,000,000
- Average reads/sec: 1B / 86,400 ≈ 11,600 /sec
- Peak read traffic (10x for viral URLs) — normal peak (2x): ~25,000 /sec
- Viral spike (10x): ~120,000 /sec
- This is our design target: 120,000 redirects/sec
Storage Estimation#
| Field | Size |
|---|---|
| Short code | 7 bytes |
| Long URL | ~200 bytes (average) |
| User ID | 8 bytes |
| Created timestamp | 8 bytes |
| Expiration timestamp | 8 bytes |
| Click count | 8 bytes |
| Metadata | ~50 bytes |
| Total per URL | ~300 bytes |
| With indexes and overhead | ~500 bytes per URL |
- URLs per day: 100M
- URLs per year: 36.5B
- URLs for 5 years: 182.5B
- Storage needed: 182.5B × 500 bytes = 91.25 TB
- With 3x replication: ~275 TB total storage
- Let's round to: 300 TB over 5 years
Key Metrics Summary#
Traffic
| Metric | Value |
|---|---|
| Write throughput | 1,200 /sec (peak: 2,500/sec) |
| Read throughput | 12,000 /sec (peak: 120,000/sec) |
| Read:Write ratio | 10:1 |
Storage
| Metric | Value |
|---|---|
| Per URL | ~500 bytes |
| Daily growth | 50 GB |
| Yearly growth | 18 TB |
| 5-year total | ~300 TB (with replication) |
Short code space
| Metric | Value |
|---|---|
| Characters | 62 (a-z, A-Z, 0-9) |
| Length | 7 characters |
| Total combinations | 62^7 = 3.5 trillion |
| URLs over 5 years | 182.5 billion |
| Utilization | ~5% (plenty of room) |
Requirements and estimates
Q1/3 Average redirect traffic is about 11,600 per second. Why does the design target 120,000 redirects per second?
Answer & explanation
Answer: A
Design for peaks, not averages. The interviewer said viral links cause 10x spikes, so 10 x ~12,000 is the target. Replication multiplies stored bytes, not the number of redirect requests.
Q2/3 Over 5 years the service needs about 182.5 billion short codes. Why 7 characters rather than 6, using 62 symbols?
Answer & explanation
Answer: B
Each extra character multiplies the space by 62. Six characters cannot cover 182.5 billion codes; seven gives about 5% utilization. 3.5 trillion is 62^7, not 62^6.
Q3/3 At about 500 bytes per URL record and 100 million new URLs per day, how fast does raw storage grow?
Answer & explanation
Answer: B
100,000,000 x 500 bytes = 50,000,000,000 bytes, about 50 GB/day, or roughly 18 TB per year before replication. 5 GB/day would mean 50-byte records, which would not even hold a 200-byte long URL.
Phase 3: High-Level Design (10 minutes)#
"Now let me sketch out the high-level architecture."
System Architecture#
%% title: High-level architecture
flowchart TD
C@{ icon: "sd:users", label: "Clients<br/>Web/Mobile/API" } --> CDN@{ icon: "tech:cloudflare", label: "CDN<br/>(CloudFlare)" }
CDN --> SA@{ icon: "tech:s3", label: "Static Assets<br/>(S3/CloudFront)" }
CDN --> LB@{ icon: "tech:elb", label: "Load Balancer<br/>(ALB)" }
LB --> API["API Service<br/>(Create)"]:::core
LB --> RS["Redirect Service<br/>(Read)"]:::hot
RS --> R1@{ icon: "tech:redis", label: "Redis Cache 1" }
RS --> R2@{ icon: "tech:redis", label: "Redis Cache 2" }
API --> PG@{ icon: "tech:postgresql", label: "PostgreSQL Cluster<br/>(Sharded)" }
R1 --> PG
R2 --> PG
PG --> AN@{ icon: "tech:kafka", label: "Analytics Pipeline<br/>(Kafka)" }
class C client
class CDN,LB edge
class SA,PG store
class R1,R2 cache
class AN queueComponent Breakdown#
"Let me walk through each component and explain why it's there."
1. CDN (CloudFlare)#
Purpose: Cache redirect responses for popular URLs at the edge
Why it matters:
- Viral URLs can get millions of hits
- CDN can serve 301/302 redirects directly
- Reduces load on our servers by 80%+ for hot URLs
- Geographic distribution reduces latency globally
Configuration:
- Cache redirects for 1 hour (send 302, or 301 with an explicit
Cache-Control: max-age; a bare 301 is cached by browsers indefinitely, hiding repeat clicks from analytics and ignoring deletes) - Use short TTL to allow for URL deletion
- Bypass cache for analytics tracking pixel
2. Load Balancer (ALB)#
Purpose: Distribute traffic and route to appropriate service
Routing rules:
POST /api/urls→ API Service (URL creation)GET /{shortCode}→ Redirect ServiceGET /api/*→ API Service (management)
3. API Service (URL Creation)#
Purpose: Handle URL creation, management, and analytics reads
Key responsibilities:
- Generate unique short codes
- Validate and sanitize long URLs
- Rate limiting per user/API key
- User session management
4. Redirect Service (Read-Heavy)#
Purpose: Handle URL redirects — the critical path
Key responsibilities:
- Look up short code → long URL
- Increment click counter (async)
- Return 301/302 redirect
- Handle 404 for non-existent URLs
Why separate services?:
- Different scaling needs (redirects are 10x writes)
- Different availability requirements
- Can optimize each service independently
5. Redis Cache#
Purpose: Cache hot URLs for fast redirects
Why Redis?:
- Sub-millisecond lookups
- Perfect for key-value (shortCode → longURL)
- Supports TTL for automatic expiration
- Cluster mode for horizontal scaling
6. PostgreSQL (Sharded)#
Purpose: Persistent storage for URL mappings
Why PostgreSQL?:
- ACID guarantees for URL uniqueness
- Mature, battle-tested
- Good partitioning support
- Rich querying for analytics
7. Analytics Pipeline (Kafka)#
Purpose: Track clicks asynchronously without slowing redirects
Flow:
- Redirect happens → publish click event to Kafka
- Consumer aggregates clicks
- Batch update to analytics store
- Dashboard reads from analytics store
Data Flow: URL Creation#
%% title: URL creation flow
flowchart TD
S1(["1 · Client sends POST /api/urls<br/>with long URL"]):::client --> S2["2 · API Service receives request"]:::core
S2 --> S3{"3 · Rate limiter checks<br/>user's quota"}:::edge
S3 -->|"exceeded"| REJ(["REJECT"]):::bad
S3 --> S4["4 · Generate unique short code<br/>(see Deep Dive 1)"]:::core
S4 --> S5@{ icon: "tech:postgresql", label: "5 · Write to PostgreSQL<br/>(with uniqueness constraint)" }
S5 -.->|"If duplicate (race condition):<br/>retry with new code"| S4
S5 --> S6@{ icon: "tech:redis", label: "6 · Write to Redis cache" }
S6 --> S7(["7 · Return short URL to client"]):::ok
class S5 store
class S6 cacheData Flow: URL Redirect#
%% title: URL redirect flow
flowchart TD
S1(["1 · Client requests GET /abc1234"]):::client --> S2{"2 · CDN checks cache"}:::edge
S2 -->|"HIT"| HIT1(["Return redirect"]):::ok
S2 -->|"MISS"| S3["3 · Redirect Service<br/>receives request"]:::core
S3 --> S4{"4 · Check Redis cache"}:::cache
S4 -->|"HIT: go to step 6"| S6
S4 -->|"MISS"| S5@{ icon: "tech:postgresql", label: "5 · Query PostgreSQL<br/>for long URL" }
S5 -->|"Not found"| NF(["Return 404"]):::bad
S5 -->|"Found: write to Redis cache (async)"| S6
S6@{ icon: "tech:kafka", label: "6 · Publish click event to Kafka<br/>(async, fire-and-forget)" } --> S7(["7 · Return 302 redirect to client"]):::ok
class S5 store
class S6 queueA redirect for /abc1234 is answered by a CDN edge if cached, else from Redis in under a millisecond, else from a PostgreSQL replica, which refills Redis; an unknown code gets a 404. Each redirect publishes a click event to Kafka without waiting for it. Creating a URL checks the rate limit (free tier: 100 requests/minute), takes the next counter value from the server's pre-allocated range (42 → bkBdhgA) with no database lookup, inserts it into PostgreSQL under a uniqueness constraint and caches it in Redis for 86,400 seconds.
High-level design
Q1/2 Why split URL creation and redirects into separate services?
Answer & explanation
Answer: C
Redirects are about 10x the traffic and target 99.99% availability; creation targets 99.9%. Separating them lets you scale the hot path without scaling everything. Both read and write the same URL data; the split is about workload, not storage.
Q2/2 A redirect must also count the click. Why publish an event to Kafka instead of running
UPDATE ... click_countin the redirect?Answer & explanation
Answer: C
The interviewer said click counts may be eventually consistent but redirects must be fast. Fire-and-forget events plus batch aggregation fit that. Exactly-once is not the reason; the win is decoupling slow writes from the fast path.
Sketch it first: unique codes without a lookup
Many API servers create URLs at the same time. How would they hand out unique 7-character codes without checking the database for each one, and without codes that look sequential? What happens to the codes a server was holding when it crashes?
One strong answer
- Pre-allocated ranges: a key service hands each server a range of 10,000 counter values with one atomic Redis
INCR; the server then uses them locally, with no coordination per code. - Not sequential-looking: scramble each counter value with a reversible permutation of the 62^7 space, then base-62 encode it.
- A crash wastes the rest of that range: a few thousand codes out of 3.5 trillion, which is negligible.
Phase 4: Deep Dives (20 minutes)#
"Great high-level design. Let's dive deeper into a few areas. How do you generate unique short codes efficiently?"
Deep Dive 1: Short Code Generation (Week 1, Day 1 — Partitioning)#
"This is a critical part of the system. Let me explain the challenge and my solution."
The Problem#
Each code must be
We need to generate 100M unique short codes per day. Each code must be:
- Unique across all time
- Unpredictable (can't guess next code)
- Fast to generate (no bottlenecks)
- Works across distributed servers
Naive approaches that fail
- Auto-increment: Predictable, single point of failure
- Random + check DB: Race conditions, slow at scale
- UUID: Too long (36 chars vs our 7)
The Solution: Pre-generated Key Ranges#
"I would use a pre-generated key range approach, combined with the partitioning concepts from our course."
Key generation service
Hands out ranges of 10,000 consecutive counter values:
- Range 1: counter values 0 – 9,999
- Range 2: counter values 10,000 – 19,999
- Range 3: counter values 20,000 – 29,999
- ...
Each counter value is scrambled with a reversible permutation of the 62^7 space and base-62 encoded to 7 characters, so codes stay unique but don't look sequential.
API servers request a range, use locally until exhausted.
| Server 1 | Server 2 | Server 3 | |
|---|---|---|---|
| Range | 0 – 9,999 | 10,000 – 19,999 | 20,000 – 29,999 |
| Next | 42 | 10,007 | 20,099 |
Implementation#
# Short Code Generation with Pre-allocated Ranges
# Applies: Week 1, Day 1 - Partitioning concepts
import threading
from dataclasses import dataclass
from typing import Optional
import redis
CHARS = "abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789"
SPACE = 62 ** 7 # ~3.5 trillion 7-character codes
MULTIPLIER = 1_580_030_173 # odd and not divisible by 31 -> coprime with 62^7
@dataclass
class KeyRange:
"""A range of pre-allocated counter values."""
current: int # Next counter value to hand out
end: int # Last counter value in range (inclusive)
def next_value(self) -> Optional[int]:
"""Get next available counter value from this range."""
if self.current > self.end:
return None # Range exhausted
value = self.current
self.current += 1
return value
def remaining(self) -> int:
"""How many values left in this range."""
return max(0, self.end - self.current + 1)
def encode(counter: int) -> str:
"""
Map a counter value to a 7-char code.
Multiplying by a number coprime with 62^7 is a bijection on [0, 62^7),
so codes stay unique but don't look sequential. (This only obscures
order; use a keyed cipher such as a Feistel network if codes must be
unguessable.)
"""
n = (counter * MULTIPLIER) % SPACE
code = ""
for _ in range(7):
code = CHARS[n % 62] + code
n //= 62
return code
class ShortCodeGenerator:
"""
Generates unique short codes using pre-allocated ranges.
Key concepts applied:
- Partitioning (Day 1): Each server gets its own range
- No coordination needed for individual code generation
- Range allocation is the only synchronized operation
"""
def __init__(self, redis_client: redis.Redis, server_id: str):
self.redis = redis_client
self.server_id = server_id
self.current_range: Optional[KeyRange] = None
self.lock = threading.Lock()
# Configuration
self.range_size = 10_000 # Codes per range
self.refill_threshold = 1_000 # Get new range when below this
def generate(self) -> str:
"""
Generate a unique short code.
This is O(1) in the common case - just increment a counter.
Only contacts central service when range is exhausted.
"""
with self.lock:
# Check if we need a new range
if self._needs_new_range():
self._allocate_new_range()
# Take next value from current range
value = self.current_range.next_value()
if value is None:
# Range exhausted mid-operation (shouldn't happen often)
self._allocate_new_range()
value = self.current_range.next_value()
return encode(value)
def _needs_new_range(self) -> bool:
"""Check if we need to allocate a new range."""
if self.current_range is None:
return True
return self.current_range.remaining() < self.refill_threshold
def _allocate_new_range(self) -> None:
"""
Allocate a new range from the central service.
Uses Redis INCR for atomic range allocation.
This is the only point of coordination.
"""
# Atomically get next range number (1, 2, 3, ...)
range_num = self.redis.incr("shortcode:range_counter")
# Range k covers counter values [(k-1) * size, k * size - 1].
# 62^7 / 10,000 = ~352 million ranges before the space runs out
# (~96 years at 100M codes/day).
start = (range_num - 1) * self.range_size
if start + self.range_size > SPACE:
raise RuntimeError("Short code space exhausted")
self.current_range = KeyRange(
current=start,
end=start + self.range_size - 1
)
# Log allocation for debugging
print(f"Server {self.server_id} allocated range starting at {start}")
# Usage Example
async def create_short_url(long_url: str, generator: ShortCodeGenerator) -> str:
"""Create a new short URL."""
# Generate unique code (O(1), no DB lookup needed)
short_code = generator.generate()
# Store mapping in database
await db.execute(
"""
INSERT INTO urls (short_code, long_url, created_at, expires_at)
VALUES ($1, $2, NOW(), NOW() + INTERVAL '5 years')
""",
short_code, long_url
)
# Cache for fast redirects
await redis.setex(f"url:{short_code}", 86400, long_url)
return f"https://tiny.link/{short_code}"Why This Approach?#
"This design applies the partitioning principles we learned:"
| Principle | Application |
|---|---|
| Partition by range | Each server owns a range of codes |
| Minimize coordination | Only range allocation needs sync |
| Even distribution | Each server takes a new range on demand (one atomic Redis INCR) when it runs low |
| No hot spots | Each server generates from its own range |
"What if a server crashes with unused codes in its range?"
"Great question. Those codes are simply lost — but that's acceptable. With 3.5 trillion possible codes and only 182 billion needed over 5 years, losing a few thousand codes per crash is negligible. The simplicity gained is worth it."
Deep Dive 2: Handling Hot URLs (Week 1, Day 4 — Hot Keys)#
"You mentioned viral URLs getting 10x traffic. How do you handle that?"
"This is exactly the hot key problem we studied. Let me show you the solution."
The Problem#
- Normal URL: 10 requests/second
- Viral URL: 100,000 requests/second
If all requests for viral URL hit the same:
- Cache server: Server overloaded
- Database shard: Shard becomes bottleneck
- Single point of failure
Examples:
- Celebrity tweets a TinyLink
- Product launch (Apple keynote links)
- News events (election results link)
The Solution: Multi-Layer Hot Key Mitigation#
- Layer 1: CDN Caching
- Cloudflare caches redirect responses at 300+ edge locations
- Hit rate for hot URLs: ~80% (Day 4's plan)
- Only ~20% of hot-URL requests reach our servers
- Layer 2: Local Server Cache
- Each API server maintains local LRU cache
- Hot URLs naturally rise to top of cache
- Eliminates Redis calls for hottest URLs
- Layer 3: Redis Read Replicas
- Multiple Redis replicas for read scaling
- Random replica selection distributes load
- Hot keys spread across all replicas
- Layer 4: Key Replication with Suffixes
- For extremely hot keys, replicate with suffixes:
url:abc1234→url:abc1234:0,url:abc1234:1,url:abc1234:2 - Client randomly picks suffix, spreading load
- For extremely hot keys, replicate with suffixes:
Implementation#
# Hot Key Handling for URL Redirects
# Applies: Week 1, Day 4 - Hot Keys and Skew
import random
import time
from typing import Optional, Tuple
import redis
from cachetools import TTLCache
class HotKeyAwareCache:
"""
Multi-layer cache with hot key detection and mitigation.
Applies Day 4 concepts:
- Detection: Track access frequency
- Mitigation: Multiple layers of caching
- Distribution: Spread hot keys across replicas
"""
def __init__(self, redis_cluster: redis.RedisCluster):
self.redis = redis_cluster
self.hot_key_threshold = 1000 # requests/minute
self.replica_count = 3 # Replicas for hot keys
# Local tracking of access patterns
self.access_counts: dict[str, int] = {}
self.hot_keys: set[str] = set()
self.last_reset = time.time()
# Layer 2: Local server cache (bounded LRU-style, short TTL so edits/deletes propagate)
self.local_cache: TTLCache = TTLCache(maxsize=10_000, ttl=5)
def get(self, short_code: str) -> Optional[str]:
"""
Get long URL for short code with hot key handling.
"""
key = f"url:{short_code}"
# Track access for hot key detection
self._track_access(key)
# Layer 2: Check local cache first
value = self.local_cache.get(key)
if value is not None:
return value
# Layer 3 & 4: Redis lookup with hot key awareness
if key in self.hot_keys:
# Hot key: use replicated keys
value = self._get_hot_key(key)
else:
# Normal key: direct lookup
value = self._get_normal_key(key)
if value is not None:
self.local_cache[key] = value
return value
def _get_normal_key(self, key: str) -> Optional[str]:
"""Normal Redis lookup."""
return self.redis.get(key)
def _get_hot_key(self, key: str) -> Optional[str]:
"""
Hot key lookup with distribution.
Hot keys are replicated with suffixes to spread load.
Client randomly selects a replica.
"""
# Randomly select replica
replica_suffix = random.randint(0, self.replica_count - 1)
replicated_key = f"{key}:{replica_suffix}"
value = self.redis.get(replicated_key)
if value is None:
# Fallback to original key (replication might be in progress)
value = self.redis.get(key)
return value
def set(self, short_code: str, long_url: str, ttl: int = 86400) -> None:
"""
Set URL mapping with hot key awareness.
"""
key = f"url:{short_code}"
# Always set the primary key
self.redis.setex(key, ttl, long_url)
# If this is a hot key, also set replicas
if key in self.hot_keys:
self._replicate_hot_key(key, long_url, ttl)
def _replicate_hot_key(self, key: str, value: str, ttl: int) -> None:
"""Create replicas of a hot key."""
pipeline = self.redis.pipeline()
for i in range(self.replica_count):
pipeline.setex(f"{key}:{i}", ttl, value)
pipeline.execute()
def _track_access(self, key: str) -> None:
"""
Track access patterns to detect hot keys.
Simple fixed-window counter (reset every minute).
In production, use a Count-Min Sketch or a top-k streaming algorithm.
"""
current_time = time.time()
# Reset counts every minute
if current_time - self.last_reset > 60:
self._analyze_and_reset()
# Increment access count
self.access_counts[key] = self.access_counts.get(key, 0) + 1
def _analyze_and_reset(self) -> None:
"""Analyze access patterns and identify hot keys."""
new_hot_keys = set()
for key, count in self.access_counts.items():
if count > self.hot_key_threshold:
new_hot_keys.add(key)
# If newly hot, create replicas
if key not in self.hot_keys:
self._promote_to_hot(key)
# Keys that cooled down
for key in self.hot_keys - new_hot_keys:
self._demote_from_hot(key)
self.hot_keys = new_hot_keys
self.access_counts = {}
self.last_reset = time.time()
def _promote_to_hot(self, key: str) -> None:
"""Promote a key to hot status by creating replicas."""
value = self.redis.get(key)
if value:
ttl = self.redis.ttl(key)
self._replicate_hot_key(key, value, max(ttl, 3600))
print(f"Promoted hot key: {key}")
def _demote_from_hot(self, key: str) -> None:
"""Remove replicas for a key that's no longer hot."""
pipeline = self.redis.pipeline()
for i in range(self.replica_count):
pipeline.delete(f"{key}:{i}")
pipeline.execute()
print(f"Demoted key: {key}")
# Redirect service using hot key aware cache
async def redirect(short_code: str, cache: HotKeyAwareCache) -> str:
"""
Handle URL redirect with hot key protection.
"""
# Try cache first (handles hot keys automatically)
long_url = cache.get(short_code)
if long_url is None:
# Cache miss - query database
long_url = await db.fetchval(
"SELECT long_url FROM urls WHERE short_code = $1",
short_code
)
if long_url is None:
raise NotFoundError(f"URL not found: {short_code}")
# Populate cache
cache.set(short_code, long_url)
# Fire-and-forget analytics event
await analytics.record_click(short_code)
return long_urlHot Key Defense Summary#
| Layer | Technology | Hit Rate | Latency |
|---|---|---|---|
| CDN | CloudFlare | ~80% for hot | <10ms |
| Local Cache | LRU in-memory | 80% for warm | <1ms |
| Redis Replicas | Redis Cluster | 99%+ | <5ms |
| Key Replication | Suffix distribution | 100% | <5ms |
"How do you handle the case where a URL goes viral suddenly?"
"The CDN provides immediate relief — it starts caching within seconds of the first request. Our hot key detection runs every minute, so there's a brief window where a new viral URL hits Redis directly. But with Redis handling 100K+ ops/sec, we can absorb the initial spike while detection kicks in."
Deep Dive 3: Database Sharding Strategy (Week 1, Day 1 — Partitioning)#
"You mentioned PostgreSQL is sharded. How do you decide which shard holds which URL?"
"This is a classic partitioning decision. Let me walk through the options and my recommendation."
Sharding Strategy Analysis#
Option 1: Hash by Short Code
shard = hash(short_code) % num_shards
Pros:
- Even distribution (hash spreads uniformly)
- Simple lookup (compute hash, go to shard)
- No hot shards (assuming no hot keys)
Cons:
- Can't query by user ("show me my URLs")
- Can't query by time ("URLs created today")
- Resharding requires moving data
Best for: Simple key-value lookups (our redirect path)
Option 2: Hash by User ID
shard = hash(user_id) % num_shards
Pros:
- All user's URLs on same shard
- Efficient "show my URLs" queries
- User operations are single-shard
Cons:
- Power users create skew (one user = millions of URLs)
- Still can't query by time
- Anonymous URLs have no user_id
Best for: User-centric applications
Option 3: Range by Time (Monthly)
shard = year_month(created_at)
2024-01→shard_2024_012024-02→shard_2024_02
Pros:
- Old data easily archived (drop old shards)
- Time-based queries efficient
- Natural data lifecycle management
Cons:
- Current month shard is HOT (all writes)
- Need to query all shards for redirect (don't know creation time)
- Uneven shard sizes if traffic varies
Best for: Time-series data, logs, analytics
My Recommendation: Hybrid Approach#
"I recommend a hybrid approach that optimizes for our two main access patterns:"
For REDIRECTS (read path)
Use consistent hashing on short_code
short_code "abc1234" → hash → shard_7
Why: Redirect needs to be fast. We know the short_code, we can compute the shard instantly.
For USER QUERIES (management path)
Secondary index table, sharded by user_id
user_urls table:
user_id(shard key)short_codecreated_at- (lightweight, just for listing)
Why: Users need to see their URLs. Keep user's data together.
The URLs live on short_code shards (for fast redirect). The user_urls index lives on user_id shards (for fast listing). Write path creates entries in both.
Implementation: Consistent Hashing#
# Database Sharding with Consistent Hashing
# Applies: Week 1, Day 1 - Partitioning strategies
import hashlib
from bisect import bisect_left
from dataclasses import dataclass
from typing import List, Dict, Any, Optional
import asyncpg
@dataclass
class ShardConfig:
"""Configuration for a database shard."""
shard_id: int
host: str
port: int
database: str
@property
def dsn(self) -> str:
return f"postgresql://{self.host}:{self.port}/{self.database}"
class ConsistentHashRing:
"""
Consistent hash ring for shard selection.
Applies Day 1 concepts:
- Consistent hashing minimizes data movement on rebalance
- Virtual nodes ensure even distribution
- Deterministic: same key always maps to same shard
"""
def __init__(self, shards: List[ShardConfig], virtual_nodes: int = 150):
self.shards = {s.shard_id: s for s in shards}
self.virtual_nodes = virtual_nodes
self.ring: List[tuple[int, int]] = [] # (hash, shard_id)
self._build_ring()
def _build_ring(self) -> None:
"""Build the consistent hash ring with virtual nodes."""
ring = []
for shard in self.shards.values():
# Create virtual nodes for each shard
for i in range(self.virtual_nodes):
# Hash the shard_id + virtual node index
key = f"{shard.shard_id}:{i}"
hash_val = self._hash(key)
ring.append((hash_val, shard.shard_id))
# Sort by hash value
ring.sort(key=lambda x: x[0])
self.ring = ring
# Extract just the hash values for binary search
self.ring_hashes = [h for h, _ in ring]
def _hash(self, key: str) -> int:
"""Compute hash value for a key."""
return int(hashlib.md5(key.encode()).hexdigest(), 16)
def get_shard(self, key: str) -> ShardConfig:
"""
Get the shard responsible for a key.
Uses binary search for O(log n) lookup.
"""
if not self.ring:
raise ValueError("No shards configured")
hash_val = self._hash(key)
# Find the first node with hash >= key's hash
idx = bisect_left(self.ring_hashes, hash_val)
# Wrap around if we're past the last node
if idx >= len(self.ring):
idx = 0
shard_id = self.ring[idx][1]
return self.shards[shard_id]
def add_shard(self, shard: ShardConfig) -> Dict[int, List[str]]:
"""
Add a new shard to the ring.
Returns mapping of shard_id -> keys that need to move.
With consistent hashing, only ~1/n keys need to move.
"""
self.shards[shard.shard_id] = shard
self._build_ring()
# In practice, you'd track which keys moved
return {}
def remove_shard(self, shard_id: int) -> Dict[int, List[str]]:
"""
Remove a shard from the ring.
Returns mapping of destination shard_id -> keys to receive.
"""
if shard_id in self.shards:
del self.shards[shard_id]
self._build_ring()
return {}
class ShardedURLDatabase:
"""
Sharded database client for URL storage.
Handles:
- Shard selection via consistent hashing
- Connection pooling per shard
- Cross-shard queries when needed
"""
def __init__(self, shard_configs: List[ShardConfig]):
self.hash_ring = ConsistentHashRing(shard_configs)
self.pools: Dict[int, asyncpg.Pool] = {}
async def initialize(self) -> None:
"""Create connection pools for all shards."""
for shard in self.hash_ring.shards.values():
self.pools[shard.shard_id] = await asyncpg.create_pool(
shard.dsn,
min_size=5,
max_size=20
)
async def create_url(
self,
short_code: str,
long_url: str,
user_id: Optional[str] = None
) -> None:
"""
Create a new URL mapping.
Writes to:
1. URL shard (keyed by short_code) - for redirects
2. User shard (keyed by user_id) - for listing
"""
# Get shard for this short_code
shard = self.hash_ring.get_shard(short_code)
pool = self.pools[shard.shard_id]
async with pool.acquire() as conn:
await conn.execute(
"""
INSERT INTO urls (short_code, long_url, user_id, created_at, expires_at)
VALUES ($1, $2, $3, NOW(), NOW() + INTERVAL '5 years')
""",
short_code, long_url, user_id
)
# Also update user index if user is known
if user_id:
await self._update_user_index(user_id, short_code)
async def get_url(self, short_code: str) -> Optional[str]:
"""
Get long URL by short code.
Single-shard operation - very fast.
"""
shard = self.hash_ring.get_shard(short_code)
pool = self.pools[shard.shard_id]
async with pool.acquire() as conn:
row = await conn.fetchrow(
"""
SELECT long_url FROM urls
WHERE short_code = $1
AND expires_at > NOW()
""",
short_code
)
return row['long_url'] if row else None
async def get_user_urls(
self,
user_id: str,
limit: int = 100
) -> List[Dict[str, Any]]:
"""
Get all URLs for a user.
Queries user index shard, then fetches details from URL shards.
"""
# Get user's shard
user_shard = self.hash_ring.get_shard(f"user:{user_id}")
pool = self.pools[user_shard.shard_id]
async with pool.acquire() as conn:
rows = await conn.fetch(
"""
SELECT short_code, created_at
FROM user_urls
WHERE user_id = $1
ORDER BY created_at DESC
LIMIT $2
""",
user_id, limit
)
# Fetch full details from URL shards (sequential here; asyncio.gather would parallelize)
urls = []
for row in rows:
url_data = await self.get_url(row['short_code'])
if url_data:
urls.append({
'short_code': row['short_code'],
'long_url': url_data,
'created_at': row['created_at']
})
return urls
async def _update_user_index(self, user_id: str, short_code: str) -> None:
"""Update the user's URL index."""
user_shard = self.hash_ring.get_shard(f"user:{user_id}")
pool = self.pools[user_shard.shard_id]
async with pool.acquire() as conn:
await conn.execute(
"""
INSERT INTO user_urls (user_id, short_code, created_at)
VALUES ($1, $2, NOW())
""",
user_id, short_code
)Shard Distribution Visualization#
- Ring positions: 0 → 90 → 180 → 270 → back to 0
- Shards on the ring: Shard 1, Shard 2, Shard 3
When a new shard is added:
- Only keys in one segment need to move
- ~1/n of total keys (not all keys)
- Minimal disruption
Deep Dive 4: Rate Limiting (Week 1, Day 3 — Rate Limiting at Scale)#
"You mentioned rate limiting for the API. How would you implement that at this scale?"
"Let me show you a distributed rate limiting approach that handles our scale."
The Problem#
API tiers
| Tier | Limit |
|---|---|
| Free | 100 requests/minute per API key |
| Starter | 1,000 requests/minute |
| Business | 10,000 requests/minute |
| Enterprise | 100,000 requests/minute |
Challenges
- Distributed servers (can't use local counters)
- Must be fast (<5ms overhead)
- Handle clock skew across servers
- Graceful degradation if rate limiter fails
Solution: Sliding Window with Redis#
# Distributed Rate Limiting
# Applies: Week 1, Day 3 - Rate Limiting at Scale
import time
from dataclasses import dataclass
from enum import Enum
from typing import Tuple, Optional
import redis
class RateLimitTier(Enum):
FREE = 100
STARTER = 1_000
BUSINESS = 10_000
ENTERPRISE = 100_000
@dataclass
class RateLimitResult:
"""Result of a rate limit check."""
allowed: bool
limit: int
remaining: int
reset_at: int # Unix timestamp when limit resets
retry_after: Optional[int] = None # Seconds to wait if blocked
class SlidingWindowRateLimiter:
"""
Sliding window rate limiter using Redis.
Applies Day 3 concepts:
- Sliding window for smooth limiting (no burst at window start)
- Redis for distributed state
- Lua script for atomic operations
"""
# Lua script for atomic rate limit check and update
RATE_LIMIT_SCRIPT = """
local key = KEYS[1]
local now = tonumber(ARGV[1])
local window = tonumber(ARGV[2])
local limit = tonumber(ARGV[3])
-- Remove old entries outside the window
redis.call('ZREMRANGEBYSCORE', key, 0, now - window)
-- Count current requests in window
local count = redis.call('ZCARD', key)
if count < limit then
-- Under limit: add this request
redis.call('ZADD', key, now, now .. ':' .. math.random())
redis.call('EXPIRE', key, window)
return {1, limit - count - 1} -- allowed, remaining
else
-- Over limit: reject
-- Get oldest entry to calculate retry time
local oldest = redis.call('ZRANGE', key, 0, 0, 'WITHSCORES')
local retry_after = 0
if #oldest > 0 then
retry_after = math.ceil(oldest[2] + window - now)
end
return {0, 0, retry_after} -- blocked, remaining, retry_after
end
"""
def __init__(self, redis_client: redis.Redis):
self.redis = redis_client
self.window_seconds = 60 # 1-minute window
# Register the Lua script
self.rate_limit_sha = self.redis.script_load(self.RATE_LIMIT_SCRIPT)
def check(
self,
api_key: str,
tier: RateLimitTier
) -> RateLimitResult:
"""
Check if request is allowed under rate limit.
Uses sliding window algorithm:
- Smoother than fixed windows (no thundering herd at window start)
- More accurate than token bucket for per-minute limits
- Single Redis round-trip with Lua script
"""
key = f"ratelimit:{api_key}"
now = time.time()
limit = tier.value
try:
result = self.redis.evalsha(
self.rate_limit_sha,
1, # Number of keys
key,
now,
self.window_seconds,
limit
)
allowed = result[0] == 1
remaining = result[1]
if allowed:
return RateLimitResult(
allowed=True,
limit=limit,
remaining=remaining,
reset_at=int(now) + self.window_seconds
)
else:
retry_after = result[2] if len(result) > 2 else self.window_seconds
return RateLimitResult(
allowed=False,
limit=limit,
remaining=0,
reset_at=int(now) + retry_after,
retry_after=retry_after
)
except redis.RedisError as e:
# Fail open: if Redis is down, allow the request
# Log and alert, but don't block users
print(f"Rate limiter error: {e}")
return RateLimitResult(
allowed=True,
limit=limit,
remaining=limit,
reset_at=int(now) + self.window_seconds
)
def get_headers(self, result: RateLimitResult) -> dict:
"""Generate conventional (X-RateLimit-*) response headers."""
headers = {
'X-RateLimit-Limit': str(result.limit),
'X-RateLimit-Remaining': str(result.remaining),
'X-RateLimit-Reset': str(result.reset_at)
}
if not result.allowed:
headers['Retry-After'] = str(result.retry_after)
return headers
# FastAPI middleware integration
from fastapi import Request, HTTPException
from fastapi.responses import JSONResponse
class RateLimitMiddleware:
"""
FastAPI middleware for rate limiting.
"""
def __init__(self, rate_limiter: SlidingWindowRateLimiter):
self.rate_limiter = rate_limiter
async def __call__(self, request: Request, call_next):
# Extract API key from header
api_key = request.headers.get('X-API-Key')
if not api_key:
# No API key = free tier with IP-based limiting
api_key = f"ip:{request.client.host}"
tier = RateLimitTier.FREE
else:
# Look up tier for this API key (cached)
tier = await self._get_tier(api_key)
# Check rate limit
result = self.rate_limiter.check(api_key, tier)
if not result.allowed:
return JSONResponse(
status_code=429,
content={
'error': 'Rate limit exceeded',
'retry_after': result.retry_after
},
headers=self.rate_limiter.get_headers(result)
)
# Process request
response = await call_next(request)
# Add rate limit headers to response
for key, value in self.rate_limiter.get_headers(result).items():
response.headers[key] = value
return response
async def _get_tier(self, api_key: str) -> RateLimitTier:
"""Look up the rate limit tier for an API key."""
# In production, this would query a cache/database
# For now, return a default
return RateLimitTier.STARTERRate Limiting Visualization#
Fixed Window Problem
Timeline (requests marked as |):
- Window 1:
[ ||||||||||] - Window 2:
[|||||||||| ]
User sends 10 at end of window 1, then 10 at start of window 2 = 20 requests in ~1 second!
Sliding Window Solution
60 second window slides with time.
Time: 0 · 30 · 60 · 90 — window at t=60 spans [0 … 60]
Counts ALL requests in last 60 seconds. No gaming the window boundary.
Deep Dive 5: Replication and Consistency (Week 1, Day 2 — Replication)#
"You mentioned PostgreSQL replication. What's your replication strategy?"
"Let me explain how we balance consistency and availability for different operations."
Replication Strategy#
%% title: Replication topology
flowchart TD
P@{ icon: "tech:postgresql", label: "Primary PostgreSQL<br/>(Writes)" }
P --> R1@{ icon: "tech:postgresql", label: "Replica 1<br/>(Sync)" }
P -.-> R2@{ icon: "tech:postgresql", label: "Replica 2<br/>(Async)" }
P -.-> R3@{ icon: "tech:postgresql", label: "Replica 3<br/>(Async)" }
R1 --- RQ["Read Queries<br/>(Load Balanced)"]:::client
R2 --- RQ
R3 --- RQ
class P core
class R1,R2,R3 storeConsistency Requirements by Operation#
# Replication and Consistency Configuration
# Applies: Week 1, Day 2 - Replication Trade-offs
from enum import Enum
from typing import Optional
import asyncpg
class ConsistencyLevel(Enum):
"""
Consistency levels for different operations.
Applies Day 2 concepts:
- Strong consistency when correctness matters
- Eventual consistency when speed matters
"""
STRONG = "strong" # Read from primary
READ_YOUR_WRITES = "ryw" # Read from the synchronous replica
EVENTUAL = "eventual" # Any replica is fine
class ReplicatedDatabase:
"""
Database client with configurable consistency levels.
"""
def __init__(
self,
primary_dsn: str,
replica_dsns: list[str],
sync_replica_dsn: str
):
self.primary_dsn = primary_dsn
self.replica_dsns = replica_dsns
self.sync_replica_dsn = sync_replica_dsn
self.primary_pool: Optional[asyncpg.Pool] = None
self.replica_pools: list[asyncpg.Pool] = []
self.sync_replica_pool: Optional[asyncpg.Pool] = None
async def initialize(self) -> None:
"""Create connection pools."""
self.primary_pool = await asyncpg.create_pool(
self.primary_dsn,
min_size=10,
max_size=50
)
for dsn in self.replica_dsns:
pool = await asyncpg.create_pool(dsn, min_size=5, max_size=20)
self.replica_pools.append(pool)
self.sync_replica_pool = await asyncpg.create_pool(
self.sync_replica_dsn,
min_size=5,
max_size=20
)
async def write(self, query: str, *args) -> None:
"""
Execute a write query on the primary.
Writes always go to primary for consistency.
"""
async with self.primary_pool.acquire() as conn:
await conn.execute(query, *args)
async def read(
self,
query: str,
*args,
consistency: ConsistencyLevel = ConsistencyLevel.EVENTUAL
):
"""
Execute a read query with specified consistency level.
Consistency levels:
- STRONG: Read from primary (freshest, but adds load)
- READ_YOUR_WRITES: Read from sync replica (needs synchronous_commit =
remote_apply; with 'on' the replica may not have replayed the write yet)
- EVENTUAL: Read from any replica (fastest, slight delay)
"""
pool = self._select_pool(consistency)
async with pool.acquire() as conn:
return await conn.fetch(query, *args)
def _select_pool(self, consistency: ConsistencyLevel) -> asyncpg.Pool:
"""Select appropriate pool based on consistency requirement."""
if consistency == ConsistencyLevel.STRONG:
return self.primary_pool
elif consistency == ConsistencyLevel.READ_YOUR_WRITES:
return self.sync_replica_pool
else: # EVENTUAL
# Pick a random async replica
import random
return random.choice(self.replica_pools)
# URL Service with appropriate consistency levels
class URLService:
"""
URL service demonstrating consistency trade-offs.
"""
def __init__(self, db: ReplicatedDatabase, cache: HotKeyAwareCache):
self.db = db
self.cache = cache
async def create_url(self, short_code: str, long_url: str) -> dict:
"""
Create a new URL.
Consistency: Strong write (must not lose data)
"""
await self.db.write(
"""
INSERT INTO urls (short_code, long_url, created_at)
VALUES ($1, $2, NOW())
""",
short_code, long_url
)
# Also cache it (eventual consistency OK here)
self.cache.set(short_code, long_url)
return {'short_code': short_code, 'long_url': long_url}
async def get_url_for_redirect(self, short_code: str) -> Optional[str]:
"""
Get URL for redirect (critical path).
Consistency: Eventual (speed matters most)
Stale read is OK: worst case, user sees old destination
for a few hundred milliseconds after an update.
"""
# Try cache first
long_url = self.cache.get(short_code)
if long_url:
return long_url
# Cache miss: query replica
rows = await self.db.read(
"SELECT long_url FROM urls WHERE short_code = $1",
short_code,
consistency=ConsistencyLevel.EVENTUAL
)
if rows:
long_url = rows[0]['long_url']
self.cache.set(short_code, long_url)
return long_url
return None
async def get_url_after_create(self, short_code: str) -> Optional[dict]:
"""
Get URL immediately after creation.
Consistency: Read-your-writes (user expects to see their URL)
After creating a URL, user immediately views it.
Must see the URL they just created.
"""
rows = await self.db.read(
"""
SELECT short_code, long_url, created_at, click_count
FROM urls WHERE short_code = $1
""",
short_code,
consistency=ConsistencyLevel.READ_YOUR_WRITES
)
if rows:
return dict(rows[0])
return None
async def get_analytics(self, short_code: str) -> Optional[dict]:
"""
Get analytics for a URL.
Consistency: Eventual (analytics can be slightly delayed)
Analytics are inherently delayed anyway, so eventual
consistency is perfect here.
"""
rows = await self.db.read(
"""
SELECT DATE(clicked_at) as date,
COUNT(*) as clicks
FROM url_clicks
WHERE short_code = $1
GROUP BY DATE(clicked_at)
ORDER BY date DESC
LIMIT 30
""",
short_code,
consistency=ConsistencyLevel.EVENTUAL
)
return {'daily_clicks': [dict(r) for r in rows]}Replication Trade-off Summary#
| Operation | Consistency | Why |
|---|---|---|
| URL Creation | Strong (primary) | Must not lose writes |
| Redirect Lookup | Eventual (any replica) | Speed is critical, stale OK |
| User's URL List | Read-your-writes | User expects to see their URLs |
| Analytics | Eventual | Already delayed, speed preferred |
| URL Deletion | Strong + cache invalidation | Must take effect immediately |
Deep dives: codes, hot URLs, sharding, consistency
Q1/4 With pre-allocated key ranges, an API server crashes holding 9,000 unused codes. What happens?
Answer & explanation
Answer: A
Range allocation needs coordination only when a server takes a new range (one Redis INCR). Wasting a few thousand codes per crash is negligible. Rolling back or reissuing would need extra coordination, and it risks duplicate codes.
Q2/4 URLs are sharded by
short_code, and a separateuser_urlsindex is sharded byuser_id. What does each create cost?Answer & explanation
Answer: B
The hybrid serves both access patterns: redirects hit one short_code shard and "my URLs" hits one user shard. The price is a second write on create. Scatter-gather is what you would need without the index.
Q3/4 A brand-new link goes viral, and hot key detection only runs once a minute. What carries the load in the meantime?
Answer & explanation
Answer: C
CDN caching kicks in on the first requests, and Redis can handle 100K+ ops/sec, so the brief detection gap is survivable. Key replication only kicks in after detection, which is why the earlier layers matter.
Q4/4 Redirect lookups use eventual consistency. Why does URL deletion need strong consistency plus cache invalidation?
Answer & explanation
Answer: A
The user deleted the link for a reason. Writing to the primary and invalidating cached copies makes it take effect immediately. Redirects can tolerate a stale destination for a moment; a link that should be gone cannot.
What breaks at the edges?
Three failures: the primary dies mid-write and the user retries; a URL expires in the database but its Redis entry lives on; every Redis node goes down. What does a user see in each case, and how do you handle it?
One strong answer
- Retry after a failed write: without care, a second URL. Send an idempotency key, and return the URL already made for it.
- Expired but cached: the link keeps redirecting. Store
expires_atin the cached value and check it on every hit; a background job cleans up. - Redis down: redirects still work, more slowly, from PostgreSQL, because Redis is a cache, not the source of truth.
Phase 5: Scaling and Edge Cases (5 minutes)#
"How would this system scale to 10x the current load?"
Scaling Strategy#
"Let me walk through the scaling vectors."
Horizontal Scaling Plan#
| Component | Current | 10x Scale |
|---|---|---|
| API Servers | 10 instances | 100 instances (stateless) |
| Redirect Servers | 20 instances | 200 instances (stateless) |
| Redis Cache | 3-node cluster | 30-node cluster |
| PostgreSQL | 5 shards | 50 shards |
| Kafka | 3 brokers | 12 brokers |
Bottleneck Analysis#
| Component | Limit | Scaling Solution |
|---|---|---|
| CDN | Virtually unlimited | CloudFlare handles automatically |
| Load Balancer | 1M conn/sec | Add more ALBs, use NLB for L4 |
| API Servers | ~500 req/s each | Add instances (stateless) |
| Redis | ~100K ops/s/node | Add nodes to cluster |
| PostgreSQL | ~10K writes/s/shard | Add more shards |
| Network | 10 Gbps | Multiple NICs, better instances |
Edge Cases#
"What edge cases should we handle?"
"Several important ones..."
Edge Case 1: Duplicate Short Code#
Scenario: Two servers generate the same short code simultaneously (shouldn't happen with our range allocation, but...)
Handling:
- Database has UNIQUE constraint on short_code
- If insert fails with duplicate error, retry with new code
- Maximum 3 retries before returning error
Code:
for attempt in range(3):
code = generator.generate()
try:
await db.create_url(code, long_url)
return code
except UniqueViolationError:
continue # Try again
raise ServiceError("Failed to generate unique code")Edge Case 2: Expired URL Still Cached#
Scenario: URL expires in database but remains in Redis cache. User can still access expired URL.
Handling:
- Include expiration time in cache value
- Check expiration on cache hit, not just DB lookup
- Background job to clean expired URLs from cache
Cache value format:
{
"long_url": "https://example.com/...",
"expires_at": 1735689600
}Edge Case 3: Database Failover During Write#
Scenario: Primary database fails mid-write. URL created but not confirmed to user.
Handling:
- Use idempotency keys (user can retry safely)
- On retry, check if URL already exists with same idempotency key
- If exists, return existing URL instead of error
Failure Scenarios#
| Failure | Detection | Impact | Recovery |
|---|---|---|---|
| Redis down | Health checks | Slower redirects (DB fallback) | Fail open, use DB directly |
| Primary DB down | Connection errors | No new URL creation | Promote sync replica to primary |
| CDN down | External monitoring | Higher origin load | DNS failover to backup CDN |
| API server crash | ALB health checks | Requests routed elsewhere | Auto-scaling replaces instance |
Phase 6: Monitoring and Operations#
"How would you monitor this system in production?"
Key Metrics#
"I would track metrics at multiple levels."
Business Metrics#
| Metric | Target | Alert |
|---|---|---|
| URLs created per second | ~1,200/sec | if <800 or >3,000 |
| Redirects per second | 12,000/sec | if <5,000 |
| 404 rate | <0.1% | if >1% |
| Unique short codes remaining | — | if <10% capacity |
System Metrics#
Service health
| Metric | Target | Current |
|---|---|---|
| Redirect latency p99 | < 50ms | 42ms |
| Create latency p99 | < 200ms | 156ms |
| Error rate | < 0.1% | 0.02% |
| Throughput | > 10K/sec | 11,247/sec |
Cache
| Metric | Target | Current |
|---|---|---|
| Redis hit rate | > 95% | 97% |
| Redis latency p99 | < 5ms | 1.2ms |
| Cache size | < 80% capacity | 62% |
Database
| Metric | Target | Current |
|---|---|---|
| Query latency p99 | < 50ms | 23ms |
| Connection pool usage | < 80% | 58% |
| Replication lag | < 100ms | 45ms |
| Disk usage | < 70% | 52% |
Alerting Strategy#
CRITICAL (PagerDuty - wake someone up)
- Redirect error rate > 5%
- Database primary unreachable
- All Redis nodes down
- Redirect latency p99 > 500ms
WARNING (Slack - business hours)
- Redirect error rate > 1%
- Cache hit rate < 90%
- Replication lag > 1 second
- API rate limit errors > 10%
INFO (Dashboard only)
- New URL creation rate changes
- Traffic pattern anomalies
- Approaching capacity thresholds
Runbook Snippet#
Symptoms
- Redirect p99 > 100ms
- User complaints about slow links
Diagnosis
- Check Redis latency:
redis-cli --latency -h redis-cluster - Check cache hit rate:
redis-cli INFO stats | grep keyspace_hits - Check database load:
SELECT * FROM pg_stat_activity WHERE state = 'active'; - Check for hot keys:
redis-cli --hotkeys
Resolution
- If Redis slow: Check memory, restart if needed
- If cache miss rate high: Check TTLs, warm cache
- If DB overloaded: Add read replicas, check slow queries
- If hot key detected: Enable key replication (see Hot Key runbook)
Escalate to: Platform team if not resolved in 15 minutes
Edge cases and operations
Q1/3 A URL expired in the database, but its Redis entry is still live, so it keeps redirecting. What is the fix?
Answer & explanation
Answer: B
Checking expiry on the cache hit closes the gap, and a background job cleans up expired entries. A one-minute TTL for every URL would wreck the hit rate just to handle a rare edge case.
Q2/3 The primary fails mid-write, the client never gets a response, and the user retries. How do you avoid creating a second URL?
Answer & explanation
Answer: B
The idempotency key ties the retry to the first attempt, so you can return the existing result instead of a duplicate. The retry gets a fresh short code, so the UNIQUE constraint would not catch it.
Q3/3 All Redis cache nodes go down. What happens to redirects in this design?
Answer & explanation
Answer: A
Redis is a cache; PostgreSQL holds the data. The design fails open to direct DB reads, with higher latency and DB load. That is also why DB capacity and replicas matter even with a high cache hit rate.
Interview Conclusion#
"Excellent work. You've demonstrated strong understanding of partitioning, replication, rate limiting, and hot key handling. I particularly liked how you connected each design decision to specific trade-offs. Any questions for me?"
"Thank you! I'd love to hear how your team currently handles URL shortening at scale, and what unexpected challenges you've faced in production."
Summary: Week 1 Concepts Applied#
Week 1 Concepts (Data at Scale)#
| Day | Concept | Application in This Design |
|---|---|---|
| Day 1 | Partitioning | Short code generation with ranges, database sharding with consistent hashing |
| Day 2 | Replication | PostgreSQL primary/replica setup, consistency levels per operation type |
| Day 3 | Rate Limiting | Sliding window rate limiter with Redis, tiered API limits |
| Day 4 | Hot Keys | Multi-layer caching, hot key detection and replication |
| Day 5 | Session Store | User session for API authentication, rate limit state |
Code Patterns Demonstrated#
| # | Pattern | Used for | Key insight |
|---|---|---|---|
| 1 | Pre-allocated key ranges | Short code generation | Minimize coordination by allocating ranges |
| 2 | Consistent hashing | Database sharding | Minimize data movement when scaling |
| 3 | Sliding window rate limiting | API rate limiting | Lua scripts for atomic Redis operations |
| 4 | Hot key replication | Handling viral URLs | Detect hot keys, replicate with suffixes |
| 5 | Consistency level selection | Database reads | Match consistency to operation requirements |
Self-Assessment Checklist#
After studying this capstone, you should be able to:
- Design a partitioning strategy based on access patterns
- Implement consistent hashing for shard selection
- Generate unique IDs with minimal central coordination
- Choose appropriate consistency levels for different operations
- Implement distributed rate limiting with Redis
- Detect and mitigate hot key problems
- Design multi-layer caching strategies
- Configure database replication for availability
- Handle edge cases like duplicate keys and failovers
- Define SLOs and monitoring strategies for a distributed system
This capstone integrates all concepts from Week 1 of the System Design Mastery Series. The URL shortener problem is a classic interview question that touches on partitioning, replication, caching, and rate limiting — all core topics for building scalable systems.
this lesson in shorts · 1
play them all →One idea from this lesson in about a minute each, made for your phone.
Go deeper · reading trail
all reading →Real write-ups from the teams that built these systems. Each opens on the publisher's site in a new tab.
Sharding Pinterest: How we scaled our MySQL fleet
Marty Weiner · 2015- What
- Pinterest's design for sharding MySQL across thousands of logical databases.
- Key idea
- Every object gets a 64-bit ID that embeds its shard ID, type and local ID, so any ID can be routed to its shard with no lookup service.
- Why here
- The capstone's short-code and sharding deep dives in one design: IDs that encode their location make reads one hop.
How Discord Stores Trillions of Messages
Bo Ingram · 2023- What
- Discord's account of hot partitions and the data services it put in front of its database.
- Key idea
- Coalescing identical in-flight reads and routing a key's requests to one service instance stops a viral object from overloading storage.
- Why here
- Directly reusable for the capstone's hot-URL deep dive: a viral short link is the same hot-key problem as a busy channel.
New lessons by email
New lessons, side quests and case files as they ship, at most once a week. No spam, unsubscribe anytime.
Discussion · …
Public: everyone reading this lesson sees comments. Be kind and keep it on the material.
