#separator:tab #html:true #tags column:3 #deck:System Design Mastery W00 · Part 1 · System Design Fundamentals
HLD vs LLD High-level design is the what: components, how they interact and why. Low-level design is the how: schemas, API contracts, algorithms, error handling. Interviews start high-level, then deep-dive one or two areas. sdm w00 foundations-part-1-basic-concepts idea W00 · Part 1 · System Design Fundamentals
The four-step framework Clarify requirements (3–5 min), estimate scale (5–7), draw the high-level design (15–20), then deep dive (10–15): schema, API, critical algorithms, edge cases and failures. sdm w00 foundations-part-1-basic-concepts idea W00 · Part 1 · System Design Fundamentals
Functional vs non-functional Functional requirements say what the system does; non-functional ones say how well: uptime, latency, throughput, consistency. The non-functional ones drive architecture, technology and cost. sdm w00 foundations-part-1-basic-concepts idea W00 · Part 1 · System Design Fundamentals
Sync vs async Sync is simple with immediate feedback, but the client blocks and services couple. Async through a queue decouples, absorbs spikes and allows retries, at the cost of eventual consistency and harder debugging. sdm w00 foundations-part-1-basic-concepts idea W00 · Part 1 · System Design Fundamentals
API styles REST for CRUD and public APIs, GraphQL when clients ask for exactly the data they need, gRPC (binary Protocol Buffers) for fast internal calls, WebSocket for server push. sdm w00 foundations-part-1-basic-concepts idea W00 · Part 1 · System Design Fundamentals
Numbers to remember

99.9% 8.76 hours of downtime a year
99.99% 52.6 minutes a year
99.999% 5.26 minutes a year
<100 · <10 · <1 ms typical P99: API response, database query, cache hit

sdm w00 foundations-part-1-basic-concepts numbers W00 · Part 1 · System Design Fundamentals
Synchronous vs asynchronous

	Synchronous	Asynchronous
Feedback	Immediate	Later (ACK on receipt)
Coupling	Tight	Loose
Traffic spike	Client waits	Queue buffers it
Use for	Login, checkout, balance check	Video processing, emails, order fan-out

sdm w00 foundations-part-1-basic-concepts tradeoff W00 · Part 1 · System Design Fundamentals
Classic mistakes

Designing before clarifying scope, scale and the read/write ratio.
Skipping non-functional requirements: they decide the architecture.
Diving into low-level detail before the high-level picture is agreed.

sdm w00 foundations-part-1-basic-concepts mistakes W00 · Part 1 · System Design Fundamentals
"Design a Twitter-like platform." What do you ask first? Scope (core feed only?), scale (100M MAU), read/write ratio (1000:1), latency (feed under 200 ms P99) and consistency (eventual is fine for a feed). Then summarize and confirm. sdm w00 foundations-part-1-basic-concepts ask W00 · Part 2 · Infrastructure Building Blocks
CDN Edge servers (PoPs) serve content near users: ~5–10 ms from a Mumbai PoP versus ~150–200 ms to a US origin. Cache static assets and identical-for-everyone responses; never user-specific data. sdm w00 foundations-part-2-infrastructure-building-blocks idea W00 · Part 2 · Infrastructure Building Blocks
Versioned URLs Put a content hash in the filename (`app.a1b2c3d4.js`) and cache it for a year as immutable. A deploy changes the URL, so users get fresh files with zero purge delay. sdm w00 foundations-part-2-infrastructure-building-blocks idea W00 · Part 2 · Infrastructure Building Blocks
no-cache doesn't mean "don't cache" `no-cache` caches but revalidates (ETag, 304 Not Modified); `no-store` never caches; `private` keeps it out of the CDN; `s-maxage` gives the CDN its own TTL. sdm w00 foundations-part-2-infrastructure-building-blocks idea W00 · Part 2 · Infrastructure Building Blocks
L4 vs L7 load balancing L4 sees IP and port: very fast (<100 μs added), any protocol. L7 sees the whole HTTP request: routes by path, host, header or cookie and terminates TLS, adding 1–5 ms. sdm w00 foundations-part-2-infrastructure-building-blocks idea W00 · Part 2 · Infrastructure Building Blocks
Shallow vs deep health checks Point the load balancer at a shallow liveness check. A deep check that queries the database marks every server unhealthy when the DB is down; use deep checks for monitoring. sdm w00 foundations-part-2-infrastructure-building-blocks idea W00 · Part 2 · Infrastructure Building Blocks
Stateless servers scale Keep no session in server memory: sessions in Redis, data in PostgreSQL, files in S3. Then any server takes any request and a crash loses nothing. sdm w00 foundations-part-2-infrastructure-building-blocks idea W00 · Part 2 · Infrastructure Building Blocks
Numbers to remember

31536000 s one year: max-age for content-hashed assets
90% hits at 1 ms vs 50 ms DB: 5.9 ms average, 8.5x faster
11 nines S3 durability: 99.999999999%
50–200 ms full DNS resolution with nothing cached

sdm w00 foundations-part-2-infrastructure-building-blocks numbers W00 · Part 2 · Infrastructure Building Blocks
Monolith vs microservices

	Monolith	Microservices
Scaling	Everything together	Per service
Failures	One affects all	Isolated
Transactions	ACID is easy	Distributed, hard
Start here when	Small team, new product	Clear boundaries, strong DevOps

sdm w00 foundations-part-2-infrastructure-building-blocks tradeoff W00 · Part 2 · Infrastructure Building Blocks
Classic mistakes

Caching user-specific or authenticated responses at the CDN.
A deep health check behind the load balancer: one database outage removes every server.
Microservices for a small team with unclear requirements; start with a monolith.

sdm w00 foundations-part-2-infrastructure-building-blocks mistakes W00 · Part 2 · Infrastructure Building Blocks
Which load balancing algorithm would you choose? Least connections by default: it adapts to real load. Round robin for identical stateless servers, weighted round robin for mixed sizes or canaries, consistent hashing for distributed caches. sdm w00 foundations-part-2-infrastructure-building-blocks ask W00 · Part 3 · Back-of-the-Envelope Estimation
Per day to per second A day is 86,400 seconds; round to 100,000. So X per day ÷ 100,000 ≈ X per second: 1 billion a day is about 10,000 a second. sdm w00 foundations-part-3-back-of-the-envelope-estimation idea W00 · Part 3 · Back-of-the-Envelope Estimation
The four estimates Traffic (RPS, read/write split, peak), storage (objects × size × retention × replication), bandwidth (RPS × payload, ingress and egress) and infrastructure (servers, replicas, cache). sdm w00 foundations-part-3-back-of-the-envelope-estimation idea W00 · Part 3 · Back-of-the-Envelope Estimation
Latency ratios matter L1 cache 0.5 ns, memory 100 ns, SSD random read 150 μs, HDD seek 10 ms; datacenter round trip 0.5 ms, cross-continent 150 ms. Cache aggressively, batch, and minimize round trips. sdm w00 foundations-part-3-back-of-the-envelope-estimation idea W00 · Part 3 · Back-of-the-Envelope Estimation
Media dominates storage For a Twitter-like service, text is 30 GB a day but images and videos add ~7 TB a day: media is 99%+ of storage. Egress for images is why the CDN is essential. sdm w00 foundations-part-3-back-of-the-envelope-estimation idea W00 · Part 3 · Back-of-the-Envelope Estimation
Numbers to remember

Peak = 2x average for normal apps; 3x social; 5–10x flash sales
5,000 RPS per server for simple APIs, at 70% utilization
30,000 queries/s per database; Redis 100,000+ ops/s
20–30% extra for database indexes; replication usually 3x

sdm w00 foundations-part-3-back-of-the-envelope-estimation numbers W00 · Part 3 · Back-of-the-Envelope Estimation
Formulas to say out loud RPS = daily actions ÷ 100,000. API servers = peak RPS ÷ 5,000 ÷ 0.7. Read replicas = read RPS ÷ 30,000. Cache = hot data: 20% of the total, or the last N days. sdm w00 foundations-part-3-back-of-the-envelope-estimation idea W00 · Part 3 · Back-of-the-Envelope Estimation
Presenting estimates

	Do	Don't
Numbers	"~12,000, call it 10K"	"12,453 per second"
Traffic	Design for peak, 2–3x	Design for the average
Time	Approximate, move on	Spend over 5 minutes
Result	Tie numbers to decisions	Skip the "so what"

sdm w00 foundations-part-3-back-of-the-envelope-estimation tradeoff W00 · Part 3 · Back-of-the-Envelope Estimation
Classic mistakes

Forgetting growth: 100 GB now can be 1 TB in a year at 10x growth.
Forgetting replication (500 GB × 3 = 1.5 TB) or that images make posts 100x bigger.
Mixing megabytes and megabits: MB/s × 8 = Mbps.

sdm w00 foundations-part-3-back-of-the-envelope-estimation mistakes W00 · Part 3 · Back-of-the-Envelope Estimation
Estimate traffic for 200M DAU each viewing 20 tweets and posting 0.5. 4 billion reads a day is about 46,000 reads per second; writes are about 1,000 per second (40:1). Peak roughly doubles, so design for ~100,000 RPS. sdm w00 foundations-part-3-back-of-the-envelope-estimation ask W00 · Part 4 · Networking Fundamentals for System Design
TCP guarantees, at a price TCP gives reliable, ordered delivery with flow and congestion control, but the handshake costs 1 RTT before the first request and one lost packet blocks everything behind it. sdm w00 foundations-part-4-networking idea W00 · Part 4 · Networking Fundamentals for System Design
UDP when late data is useless No handshake, an 8-byte header, message boundaries kept, no reliability. Use it for real-time video, voice, gaming, DNS and as QUIC's base. sdm w00 foundations-part-4-networking idea W00 · Part 4 · Networking Fundamentals for System Design
HTTP/1.1 → 2 → 3 HTTP/1.1 sends one request at a time per connection. HTTP/2 multiplexes streams and compresses headers over one TCP connection. HTTP/3 runs QUIC over UDP: a lost packet only blocks its own stream. sdm w00 foundations-part-4-networking idea W00 · Part 4 · Networking Fundamentals for System Design
Reuse connections Each new connection pays TCP and TLS handshakes and starts in slow start. Keep-alive and connection pools avoid that; short connections also pile up in TIME_WAIT and exhaust ephemeral ports. sdm w00 foundations-part-4-networking idea W00 · Part 4 · Networking Fundamentals for System Design
Scaling WebSockets Connections are stateful, so the load balancer needs stickiness, and users on different servers reach each other through a pub/sub backbone (Redis, Kafka). Use SSE for one-way server push. sdm w00 foundations-part-4-networking idea W00 · Part 4 · Networking Fundamentals for System Design
Numbers to remember

1 RTT TCP handshake; TLS 1.3 adds 1, TLS 1.2 adds 2
~55 ms minimum NYC–London RTT, set by light in fiber
0.5 · 1–2 · 100–200 ms same datacenter, cross-AZ, cross-continent
30,000 ports needed at 500 connections/s × 60 s TIME_WAIT

sdm w00 foundations-part-4-networking numbers W00 · Part 4 · Networking Fundamentals for System Design
gRPC vs REST

	gRPC	REST
Format	Protobuf, binary	JSON, text
Browser support	Limited (grpc-web)	Native
Streaming	Full	Limited
Use for	Internal microservices	Public APIs, browsers

sdm w00 foundations-part-4-networking tradeoff W00 · Part 4 · Networking Fundamentals for System Design
Classic mistakes

Opening a new connection per request instead of pooling: handshakes, slow start, port exhaustion.
WebSocket for request-response or one-way push, where HTTP or SSE is simpler.
Assuming HTTP/2 solved head-of-line blocking: it still rides one TCP connection.

sdm w00 foundations-part-4-networking mistakes W00 · Part 4 · Networking Fundamentals for System Design
Why does TCP use a three-way handshake, not two? It proves both sides can send and receive. With two steps the server can't know the client got its SYN-ACK, or tell a new SYN from a delayed duplicate. sdm w00 foundations-part-4-networking ask W00 · Part 5 · Operating System Concepts for System Design
Processes isolate, threads share A process owns its memory and file descriptors, so one crash kills only itself. Threads share the heap and descriptors with their own stacks: cheap to create and switch, but one crash kills them all. sdm w00 foundations-part-5-operating-systems idea W00 · Part 5 · Operating System Concepts for System Design
Event loops scale connections Thread-per-connection doesn't scale past ~10K connections. epoll registers descriptors once and returns only the ready ones, so one thread serves thousands (Node.js, nginx, Redis). sdm w00 foundations-part-5-operating-systems idea W00 · Part 5 · Operating System Concepts for System Design
Context switches cost more than they look Saving registers is cheap; the TLB flush, cold CPU caches and pipeline flush are not. Use async I/O, thread pools, batching and CPU affinity to switch less. sdm w00 foundations-part-5-operating-systems idea W00 · Part 5 · Operating System Concepts for System Design
Write-ahead logging Append the change to a sequential log and fsync it before touching data files; on a crash, replay the log. Sequential HDD writes run ~100 MB/s versus ~1 MB/s random. sdm w00 foundations-part-5-operating-systems idea W00 · Part 5 · Operating System Concepts for System Design
Containers are namespaces plus cgroups Namespaces isolate what a process sees (PID, network, mounts, users); cgroups limit what it uses (CPU quota, memory). Exceed the memory limit and the container is OOM killed. sdm w00 foundations-part-5-operating-systems idea W00 · Part 5 · Operating System Concepts for System Design
Numbers to remember

1–10 μs vs 10–100 μs context switch: threads vs processes
~1 MB stack per thread: 1M threads need 1 TB of stacks
1,024 default file descriptor limit; each connection uses one
1.5–2x slower remote-node memory access on NUMA systems

sdm w00 foundations-part-5-operating-systems numbers W00 · Part 5 · Operating System Concepts for System Design
Mutex vs semaphore vs RWLock

	Mutex	Semaphore	RWLock
Lets in	One thread	N threads	Many readers or one writer
Ownership	Locker unlocks	None	Per lock mode
Use for	Shared data	Pools, rate limits	Read-heavy data

sdm w00 foundations-part-5-operating-systems tradeoff W00 · Part 5 · Operating System Concepts for System Design
Classic mistakes

Thread-per-connection for a million connections instead of an event loop.
Taking locks in different orders across threads: a deadlock waiting to happen.
Running a database that caches pages itself without O_DIRECT, so data is cached twice.

sdm w00 foundations-part-5-operating-systems mistakes W00 · Part 5 · Operating System Concepts for System Design
What happens when a process touches memory that isn't in RAM? A page fault. Minor: in memory but unmapped, the OS updates the page table (~1 μs). Major: read from disk (~10 ms on HDD). Invalid: SIGSEGV. sdm w00 foundations-part-5-operating-systems ask W00 · Part 6 · System Design Terminology & Theory
CAP is C or A during a partition Partitions will happen, so P isn't optional. During one, a CP system refuses what it can't confirm (ZooKeeper, etcd); an AP system answers with possibly stale data (Cassandra, DNS). sdm w00 foundations-part-6-terminology-and-theory idea W00 · Part 6 · System Design Terminology & Theory
PACELC Even with no partition, strong consistency costs latency: synchronous replication waits for the slowest replica. DynamoDB and Cassandra are PA/EL; HBase is PC/EC. sdm w00 foundations-part-6-terminology-and-theory idea W00 · Part 6 · System Design Terminology & Theory
Isolation levels Read uncommitted allows dirty reads; read committed (PostgreSQL's default) allows non-repeatable reads; repeatable read (MySQL's default) allows phantoms; serializable allows none. sdm w00 foundations-part-6-terminology-and-theory idea W00 · Part 6 · System Design Terminology & Theory
Consistency models Linearizability respects real-time order; sequential keeps only each process's program order; causal keeps cause before effect (vector clocks); eventual only promises convergence. sdm w00 foundations-part-6-terminology-and-theory idea W00 · Part 6 · System Design Terminology & Theory
Delivery semantics At-most-once may lose messages; at-least-once may duplicate them, so consumers must be idempotent; exactly-once needs idempotent producers and consumers, or transactional messaging. sdm w00 foundations-part-6-terminology-and-theory idea W00 · Part 6 · System Design Terminology & Theory
Error budgets A 99.9% SLO allows 43.2 minutes of downtime in a 30-day month. Budget left: ship fast. Budget gone: freeze features and fix reliability. Keep the SLA looser than the SLO. sdm w00 foundations-part-6-terminology-and-theory idea W00 · Part 6 · System Design Terminology & Theory
Numbers to remember

R + W > N quorum overlap, e.g. N=3, W=2, R=2
3f + 1 nodes to tolerate f Byzantine failures
43.2 min monthly error budget at 99.9%
~10 ms an fsync on HDD; group commit batches them

sdm w00 foundations-part-6-terminology-and-theory numbers W00 · Part 6 · System Design Terminology & Theory
ACID vs BASE

	ACID	BASE
Consistency	Strong	Eventual
Approach	Pessimistic: lock early	Optimistic: resolve later
Scaling	Mainly vertical	Horizontal
Use for	Payments, inventory, accounts	Feeds, catalogs, sessions

sdm w00 foundations-part-6-terminology-and-theory tradeoff W00 · Part 6 · System Design Terminology & Theory
Classic mistakes

"Pick 2 of 3" in CAP: partition tolerance is a given, not a choice.
Assuming exactly-once delivery is free instead of making consumers idempotent.
Saga compensations that aren't idempotent: they may run more than once.

sdm w00 foundations-part-6-terminology-and-theory mistakes W00 · Part 6 · System Design Terminology & Theory
"Explain CAP and how it applies to your design." Different parts choose differently. Profiles: AP, since a bio stale for seconds beats an error; resolve conflicts with last-write-wins or version vectors. Payments: CP, an error beats a wrong balance. sdm w00 foundations-part-6-terminology-and-theory ask W00 · Part 7 · API Design for System Design
Resources, not verbs Model things as plural nouns with a few standard methods: Get, List, Create, Update, Delete. A real verb stays rare and attached to a resource: `POST /v1/orders/{id}:cancel`. sdm w00 foundations-part-7-api-design idea W00 · Part 7 · API Design for System Design
Paginate with cursors from day one Adding pagination later breaks clients. Keyset on (sort key, unique ID) with an opaque token costs the same on every page; offset walks every skipped row and duplicates or skips rows under writes. sdm w00 foundations-part-7-api-design idea W00 · Part 7 · API Design for System Design
Additive changes only Within a version, add endpoints, optional request fields and response fields. Renames, removals, type changes, new required fields and changed defaults break clients. Clients ignore unknown fields. sdm w00 foundations-part-7-api-design idea W00 · Part 7 · API Design for System Design
Idempotency keys on POST The client sends one key per logical operation and reuses it on every retry. A repeat returns the original response; the same key with a different body is an error. sdm w00 foundations-part-7-api-design idea W00 · Part 7 · API Design for System Design
Errors are a contract Clients branch on a stable code; people read the message. RFC 9457 problem details, plus the field and whether to retry. One shape for every error, including the gateway's 429s. sdm w00 foundations-part-7-api-design idea W00 · Part 7 · API Design for System Design
Slow work becomes a resource Return `202` with a status resource to poll, whose terminal states never change. Servers also get a webhook, mobile apps a push. Big results are a link to object storage. sdm w00 foundations-part-7-api-design idea W00 · Part 7 · API Design for System Design
Numbers to remember

100 max page size: Stripe `limit`, most GitHub `per_page`
24 hours before Stripe may prune an idempotency key
5,000 an hour GitHub REST limit per user; 60 unauthenticated
24 months GitHub keeps supporting the previous version

sdm w00 foundations-part-7-api-design numbers W00 · Part 7 · API Design for System Design
Offset vs cursor pagination

	Offset	Cursor
Cost	Grows with depth	Constant per page
Concurrent writes	Duplicates and gaps	Stable
Jump to page 50	Yes	No
Fits	Small bounded lists	Feeds, histories, public APIs

sdm w00 foundations-part-7-api-design tradeoff W00 · Part 7 · API Design for System Design
Classic mistakes

POSTs without an idempotency key: every timeout on an order or payment can become a duplicate.
Renaming or retyping a field inside a version instead of adding alongside.
Slow work inside the request instead of `202` and an operation.

sdm w00 foundations-part-7-api-design mistakes W00 · Part 7 · API Design for System Design
POST /orders timed out. Did the order go through? Retry with the same idempotency key: the server returns the original `201` if the first landed, or creates the order once. A changed cart needs a new key. sdm w00 foundations-part-7-api-design ask W00 · Part 8 · Databases in Depth
A commit is one sequential write Every change is appended to the WAL first; a commit is durable once its log record is flushed, while changed pages are written later. Group commit lets one fsync cover many transactions. sdm w00 foundations-part-8-databases-in-depth idea W00 · Part 8 · Databases in Depth
Composite index order Equality columns first, then the one range or sort column. `(user_id, created_at, id)` reads 20 adjacent entries for a user's latest orders; `INCLUDE` the payload to make it covering. sdm w00 foundations-part-8-databases-in-depth idea W00 · Part 8 · Databases in Depth
Lost update and write skew Read-then-write code loses updates on both PostgreSQL's and InnoDB's defaults: use a guarded atomic update or `FOR UPDATE`. Write skew needs Serializable or a constraint like `UNIQUE (customer_id, coupon_code)`. sdm w00 foundations-part-8-databases-in-depth idea W00 · Part 8 · Databases in Depth
Where MVCC keeps old versions PostgreSQL writes a new tuple in the heap and VACUUM removes the old one. InnoDB updates in place and keeps old versions in undo, which purge discards. A long transaction blocks both. sdm w00 foundations-part-8-databases-in-depth idea W00 · Part 8 · Databases in Depth
Optimistic concurrency Each row has a `version`; the update runs `WHERE version = seen`. Zero rows means someone changed it: reload and decide again. It spans a user's think time, and maps to `ETag` and 412. sdm w00 foundations-part-8-databases-in-depth idea W00 · Part 8 · Databases in Depth
Time-ordered IDs UUIDv4 inserts at random index leaves, so the whole index becomes the working set. UUIDv7 and Snowflake IDs sort by time and keep inserts at the index's right edge. sdm w00 foundations-part-8-databases-in-depth idea W00 · Part 8 · Databases in Depth
Numbers to remember

8 kB · 16 KB page size: PostgreSQL, InnoDB default
4 B-tree depth for a billion rows, whatever the key
~1,000 commits a second, each waiting for its own 1 ms flush
25% of RAM suggested PostgreSQL `shared_buffers`, dedicated server

sdm w00 foundations-part-8-databases-in-depth numbers W00 · Part 8 · Databases in Depth
B-tree vs LSM-tree

	B-tree	LSM-tree
Writes	In place, random, on commit path	Appends, merged in background
Reads	~1 disk read, predictable	Several files; scans cost more
Best for	Read-heavy OLTP, ranges	Messages, events, metrics
Examples	PostgreSQL, InnoDB	RocksDB, Cassandra, ScyllaDB

sdm w00 foundations-part-8-databases-in-depth tradeoff W00 · Part 8 · Databases in Depth
Classic mistakes

Trusting the default isolation level with read-then-write code: neither prevents lost updates.
An index for every slow query. Each taxes every write; read `EXPLAIN` first.
Long transactions on the primary: they block vacuum and purge. Run reports on a replica.

sdm w00 foundations-part-8-databases-in-depth mistakes W00 · Part 8 · Databases in Depth
How do you stop double-selling the last item? `SET stock = stock - 1 WHERE id = ? AND stock > 0`, then check rows affected: zero means sold out. Use `FOR UPDATE` if logic sits between the read and the write. sdm w00 foundations-part-8-databases-in-depth ask W01 · Day 1 · Partitioning Deep-Dive
Partitioning is how you scale writes Vertical scaling hits a ceiling and read replicas don't fix write bottlenecks. Partitioning (sharding) splits data across machines, each owning a subset, so storage, throughput and memory add up. sdm w01 day-01-partitioning-deep-dive idea W01 · Day 1 · Partitioning Deep-Dive
Hash partitioning `hash(key) % N` spreads keys evenly with an O(1) lookup, but range queries become scatter-gather across every partition, and changing N moves most keys. sdm w01 day-01-partitioning-deep-dive idea W01 · Day 1 · Partitioning Deep-Dive
Range partitioning Contiguous key ranges keep a range query on one partition, but auto-increment IDs or timestamps send every new write to the last partition. sdm w01 day-01-partitioning-deep-dive idea W01 · Day 1 · Partitioning Deep-Dive
Directory partitioning A lookup table maps keys to partitions: any key can go anywhere and rebalancing is a directory update, but the directory is a single point of failure and adds a lookup to every request. sdm w01 day-01-partitioning-deep-dive idea W01 · Day 1 · Partitioning Deep-Dive
Consistent hashing Keys and nodes sit on a ring; a key belongs to the first node clockwise. Adding a node moves only about 1/(N+1) of keys, from one neighbour. Virtual nodes even out the load. sdm w01 day-01-partitioning-deep-dive idea W01 · Day 1 · Partitioning Deep-Dive
Numbers to remember

~80% of keys move from 4 to 5 partitions with `hash % N`
~99% move going from 100 to 101
~50% move when doubling 4 to 8, modulo's best case
1 GB · 100 QPS per partition minimum, or you're over-partitioned

sdm w01 day-01-partitioning-deep-dive numbers W01 · Day 1 · Partitioning Deep-Dive
Hash vs range partitioning

	Hash	Range
Distribution	Even	Skewed if keys aren't uniform
Range queries	Scatter-gather	One partition
Hot spot	One viral key	The newest range
Resharding	Modulo moves most keys	Split or merge a range

sdm w01 day-01-partitioning-deep-dive tradeoff W01 · Day 1 · Partitioning Deep-Dive
Classic mistakes

Partitioning by `hash(user_id + timestamp)`: perfect spread, but no more queries by user_id alone.
Range partitions on auto-increment IDs or time, so every new write lands on one partition.
Over-partitioning early: 1000 partitions × 100 app servers = 100,000 connections.

sdm w01 day-01-partitioning-deep-dive mistakes W01 · Day 1 · Partitioning Deep-Dive
Would you use consistent hashing for a URL shortener? Start with simple modulo hashing: scaling events are rare and planned, and the migration is manageable. Move to consistent hashing if scaling becomes frequent or migration windows unaffordable. Don't optimize prematurely. sdm w01 day-01-partitioning-deep-dive ask W01 · Day 2 · Replication Trade-offs
Three reasons to replicate Replicas buy high availability (fail over to a copy), read scaling (reads spread across copies; writes don't) and geographic latency (read from a nearby copy). sdm w01 day-02-replication-tradeoffs idea W01 · Day 2 · Replication Trade-offs
Sync, async, semi-sync Sync waits for followers: no loss, slower writes. Async acks at once: fast, but a dead leader loses unreplicated writes. Semi-sync waits for one follower: the production middle ground. sdm w01 day-02-replication-tradeoffs idea W01 · Day 2 · Replication Trade-offs
Quorums With N replicas, if W + R > N every read overlaps the latest write. N=3 with W=2, R=2 is strongly consistent; W=1, R=1 is faster but only eventual. sdm w01 day-02-replication-tradeoffs idea W01 · Day 2 · Replication Trade-offs
Replication lag anomalies Lagging followers break read-your-writes (your update vanishes), monotonic reads (data goes back in time) and consistent prefix (a reply shows before its message). sdm w01 day-02-replication-tradeoffs idea W01 · Day 2 · Replication Trade-offs
Split brain needs fencing A network partition can leave two leaders accepting writes. Fence the old one: STONITH, storage or network fencing, or a lease it must keep renewing in ZooKeeper or etcd. sdm w01 day-02-replication-tradeoffs idea W01 · Day 2 · Replication Trade-offs
Numbers to remember

99.9% one server: ~8.8 h down a year; two independent: 99.9999%, ~31 s
~200 km/ms light in fiber; Tokyo to Virginia ~55 ms each way
10–60 s failover window in which writes fail
RF=3 standard: survives one failure, quorum intact

sdm w01 day-02-replication-tradeoffs numbers W01 · Day 2 · Replication Trade-offs
Leader-follower vs multi-leader vs leaderless

	Leader-follower	Multi-leader	Leaderless
Writes go to	One leader	Any regional leader	W nodes
Conflicts	None	LWW, merge, app logic	Timestamps, read repair
Breaks on	Leader death, lag	Conflict resolution	Clock skew, tail latency

sdm w01 day-02-replication-tradeoffs tradeoff W01 · Day 2 · Replication Trade-offs
Classic mistakes

Serving a balance, inventory count or permission check from an async replica.
Failing over without fencing the old primary: two primaries, conflicting data.
Last-write-wins across leaders: clocks disagree and writes silently vanish.

sdm w01 day-02-replication-tradeoffs mistakes W01 · Day 2 · Replication Trade-offs
A friend clicks a URL right after it's created. What happens with async replication? A lagging replica may return 404, though lag is typically under 100 ms. Write-through cache the URL at creation so redirects hit the cache first. sdm w01 day-02-replication-tradeoffs ask W01 · Day 3 · Rate Limiting at Scale
Limits must be counted together With 50 servers each seeing a slice of a user's traffic, uncoordinated local counters let 100 requests through a 10-per-minute limit. Counts must be shared, synced or sticky. sdm w01 day-03-rate-limiting-at-scale idea W01 · Day 3 · Rate Limiting at Scale
Fixed window's boundary burst One counter per window is simple and O(1), but a user can spend a full quota just before and just after the boundary: 2x the limit within seconds. sdm w01 day-03-rate-limiting-at-scale idea W01 · Day 3 · Rate Limiting at Scale
Sliding window counter Weight the previous window by its overlap and add the current one: (8 × 75%) + 3 = 9. Two counters per key, no boundary burst; the default for API limits. sdm w01 day-03-rate-limiting-at-scale idea W01 · Day 3 · Rate Limiting at Scale
Token bucket allows controlled bursts Tokens refill at a fixed rate up to a capacity and each request spends one: the long-term rate is bounded, bursts up to capacity pass. Compute refill from elapsed time. sdm w01 day-03-rate-limiting-at-scale idea W01 · Day 3 · Rate Limiting at Scale
Fail open or fail closed When the limit store dies, over-allow user-facing and paid APIs; under-allow DDoS protection, cost control and login. A conservative local fallback sits in between. sdm w01 day-03-rate-limiting-at-scale idea W01 · Day 3 · Rate Limiting at Scale
Numbers to remember

2x effective rate at a fixed-window boundary
+1–2 ms per check against a central Redis counter
~33% of keys unreachable when one node of a 3-node Redis cluster dies
10–30 s Redis Cluster automatic failover

sdm w01 day-03-rate-limiting-at-scale numbers W01 · Day 3 · Rate Limiting at Scale
Distributed counting

	Central Redis	Local + sync	Sticky sessions
Accuracy	High	Approximate	High per server
Latency	+1–2 ms	Very low	Very low
On failure	Total failure if Redis is down	Continues on local limits	Failover disruption

sdm w01 day-03-rate-limiting-at-scale tradeoff W01 · Day 3 · Rate Limiting at Scale
Classic mistakes

Each server counting alone: users spread requests and multiply the limit.
One blanket fail-open rule: an AI call costing $0.01 each should fail closed.
Leaky bucket for user-facing limits: it queues requests and adds latency even with spare capacity.

sdm w01 day-03-rate-limiting-at-scale mistakes W01 · Day 3 · Rate Limiting at Scale
One Redis node dies. What happens to rate limiting? About 33% of keys are unreachable during the 10–30 s failover. Fall back to a conservative local limit, accept a brief over-allow, alert and log fallback mode. sdm w01 day-03-rate-limiting-at-scale ask W01 · Day 4 · Hot Keys and Skew
Traffic follows Zipf's law The nth most popular key gets about 1/n of the top key's traffic. With 1,000 keys the top 1% take ~40%; viral content pushes the top 0.01% past 90%. sdm w01 day-04-hot-keys-and-skew idea W01 · Day 4 · Hot Keys and Skew
Detect with a Count-Min Sketch Fixed memory, O(depth) per update: hash the key into several counters and read the minimum. It may overestimate but never underestimates, which is all hot key detection needs. sdm w01 day-04-hot-keys-and-skew idea W01 · Day 4 · Hot Keys and Skew
Layer the defenses for hot reads CDN edge, then local app memory, then Redis with split keys and request coalescing; the database sees only misses. In the lesson, traffic past the edge drops from 10,000 to ~2,800 req/s. sdm w01 day-04-hot-keys-and-skew idea W01 · Day 4 · Hot Keys and Skew
Write-heavy hot keys can't be cached Buffer increments and flush periodically, shard the counter across N keys (reads sum all N), or use a G-Counter CRDT where each node owns its own count. sdm w01 day-04-hot-keys-and-skew idea W01 · Day 4 · Hot Keys and Skew
Bifurcate by popularity Celebrities get their own code path: posts from accounts under 10K followers are pushed into feeds; celebrity posts are pulled at read time and merged. Show approximate counts like "1.2M likes". sdm w01 day-04-hot-keys-and-skew idea W01 · Day 4 · Hot Keys and Skew
Numbers to remember

0.01% of URLs taking 90% of traffic in the redesign
>1000x average key traffic, or >10% of a partition: problematically hot
P99 500 → 20 ms after multi-layer hot key handling
10K followers push on write below, pull on read above

sdm w01 day-04-hot-keys-and-skew numbers W01 · Day 4 · Hot Keys and Skew
Hot key strategies

	Best for	Consistency	Write cost
Caching	Read-heavy, stale OK	Eventual	None
Key splitting	Read-heavy, rare writes	Weak	High fan-out
Local cache	Ultra-low latency	Weak	Invalidation
Coalescing	Misses, cold starts	Strong	None

sdm w01 day-04-hot-keys-and-skew tradeoff W01 · Day 4 · Hot Keys and Skew
Classic mistakes

Assuming uniform traffic: one viral key turns a partition into a single-node bottleneck.
Letting a hot key expire all at once: thousands of requests stampede the database.
Splitting keys that are written often: every write fans out to all splits.

sdm w01 day-04-hot-keys-and-skew mistakes W01 · Day 4 · Hot Keys and Skew
How do you choose between caching, replication and sharding for a hot key? Caching first: it solves 90% of hot key problems. Read replicas when reads need freshness. Sharded counters or batching only when writes are the bottleneck. sdm w01 day-04-hot-keys-and-skew ask W01 · Day 5 · Session Store Design
Sessions are read-heavy and forgiving Every request validates a session (80% reads), so latency must stay under 5 ms P99. Durability is best effort: a lost session means logging in again, inconvenient but not catastrophic. sdm w01 day-05-session-store-design idea W01 · Day 5 · Session Store Design
Sticky vs distributed sessions Sticky sessions keep state in one server's memory: fastest, but a server failure logs its users out and deploys lose sessions. A distributed store (Redis) costs a network hop and survives both. sdm w01 day-05-session-store-design idea W01 · Day 5 · Session Store Design
Client-side sessions can't be revoked A JWT or encrypted cookie needs no storage and scales anywhere, but you can't invalidate it before expiry, it's capped at 4 KB and rides on every request. sdm w01 day-05-session-store-design idea W01 · Day 5 · Session Store Design
Region-affine sessions Encode the home region in the session ID (`sess_east_abc123`) and route there. If a region dies, its users re-authenticate; make that painless with SSO or remember-me. sdm w01 day-05-session-store-design idea W01 · Day 5 · Session Store Design
Session security basics 256-bit random IDs, a fresh ID on login to block fixation, and cookies set HttpOnly, Secure and SameSite=Lax. sdm w01 day-05-session-store-design idea W01 · Day 5 · Session Store Design
Numbers to remember

10M × 2 KB = 20 GB fits in memory on a few servers
2–3M reads/s realistic peak: 20–30% of sessions active
6 shards × 2 = 12 nodes the Redis Cluster plan, ~3.3 GB a shard
16384 Redis Cluster hash slots

sdm w01 day-05-session-store-design numbers W01 · Day 5 · Session Store Design
Redis Cluster vs DynamoDB for sessions

	Redis Cluster	DynamoDB
Latency	Sub-ms (0.2–0.5 ms)	5–10 ms
Durability	Memory-only by default	Replicated across AZs
Operations	You run the cluster	Fully managed
Multi-region writes	Extra tooling	Global Tables

sdm w01 day-05-session-store-design tradeoff W01 · Day 5 · Session Store Design
Classic mistakes

Local session caches without short TTLs or pub/sub invalidation: logged-out sessions keep working.
A `user_sessions` set without the sessions' TTL: it fills with expired IDs.
Sequential, guessable session IDs instead of secure random ones.

sdm w01 day-05-session-store-design mistakes W01 · Day 5 · Session Store Design
How would you implement "log out from all devices"? Keep a `user_sessions:{user_id}` set beside each `session:{id}` key. On logout-all, read the set and delete every session in one pipeline. Alternative: bump a per-user session version. sdm w01 day-05-session-store-design ask W01 · Boss · Design a URL Shortener at Scale
Clarify, then estimate Ask before designing: spikes, code alphabet, duplicate URLs, analytics freshness, API rate limits. The answers set the target: 120,000 redirects/s during viral spikes. sdm w01 capstone-interview-week-1-url-shortner-at-scale idea W01 · Boss · Design a URL Shortener at Scale
Pre-allocated key ranges A key service hands each API server a range of 10,000 counter values, scrambled and base-62 encoded to 7 characters. Only range allocation needs coordination; a crash just loses a few thousand codes. sdm w01 capstone-interview-week-1-url-shortner-at-scale idea W01 · Boss · Design a URL Shortener at Scale
Split redirect from creation Redirects are 10x writes and need 99.99% uptime, creation 99.9%. Separate services scale and fail independently, and clicks go to Kafka asynchronously so counting never slows a redirect. sdm w01 capstone-interview-week-1-url-shortner-at-scale idea W01 · Boss · Design a URL Shortener at Scale
Hybrid sharding URLs live on shards chosen by consistent hashing on short_code (fast redirects); a light `user_urls` index sharded by user_id serves "my URLs". Every create writes both. sdm w01 capstone-interview-week-1-url-shortner-at-scale idea W01 · Boss · Design a URL Shortener at Scale
Numbers to remember

62^7 ≈ 3.5 trillion 7-character codes; 182.5B needed in 5 years
~1,160 / 11,600 per s average creates / redirects
~500 bytes per URL with indexes; ~300 TB over 5 years replicated
~80% hot-URL traffic answered at the CDN edge

sdm w01 capstone-interview-week-1-url-shortner-at-scale numbers W01 · Boss · Design a URL Shortener at Scale
Consistency by operation

	Consistency	Why
Create URL	Strong (primary)	Must not lose writes
Redirect	Eventual (any replica)	Speed is critical
My URLs list	Read-your-writes	Users expect their URLs
Delete URL	Strong + cache invalidation	Must take effect now

sdm w01 capstone-interview-week-1-url-shortner-at-scale tradeoff W01 · Boss · Design a URL Shortener at Scale
Classic mistakes

Auto-increment codes (predictable), random-then-check (races) or UUIDs (36 characters, not 7).
A bare 301 redirect: browsers cache it forever, hiding clicks and ignoring deletes.
Caching URLs without their expiry, so expired links keep redirecting.

sdm w01 capstone-interview-week-1-url-shortner-at-scale mistakes W01 · Boss · Design a URL Shortener at Scale
What if a server crashes with unused codes in its range? They're lost, and that's fine: with 3.5 trillion possible codes and 182 billion needed over 5 years, a few thousand per crash is negligible. Simplicity wins. sdm w01 capstone-interview-week-1-url-shortner-at-scale ask W02 · Day 1 · Timeout Hell
Slow is worse than down A down service fails fast; a slow one holds your threads. With 100 threads, a 10 s downstream cuts you from 1,000 to 10 requests a second. Timeouts turn slow into down. sdm w02 day-01-timeout-hell idea W02 · Day 1 · Timeout Hell
Percentiles, not averages Set a timeout at the ~65 ms average and 5% of healthy requests fail. Rule of thumb: timeout = P99 × 2–3, so P99 = 200 ms gives 400–600 ms. sdm w02 day-01-timeout-hell idea W02 · Day 1 · Timeout Hell
Timeout budgets Give each dependency a share of the SLA, not the whole SLA. For a 5 s payment: 4.5 s budget, fraud check 600 ms, bank 3.5 s, notification (optional) 300 ms: 4.4 s in all. sdm w02 day-01-timeout-hell idea W02 · Day 1 · Timeout Hell
Deadline propagation Pass the absolute deadline down the chain, so each hop waits only for the time left. Per-hop 5 s timeouts across three services can add up to 15 s after the user gave up at 5. sdm w02 day-01-timeout-hell idea W02 · Day 1 · Timeout Hell
Connection vs read timeout Connection timeout covers the TCP handshake and should be short (1–5 s): if you can't connect, something is badly wrong. Read timeout covers the work and depends on the operation. sdm w02 day-01-timeout-hell idea W02 · Day 1 · Timeout Hell
Numbers to remember

P99 × 2–3 the timeout rule of thumb
90% of the SLA as the total budget, e.g. 4.5 s of 5 s
5 vs 15+ min bank-slowdown impact with vs without a circuit breaker

sdm w02 day-01-timeout-hell numbers W02 · Day 1 · Timeout Hell
Fixed vs adaptive timeouts

	Fixed	Adaptive
Behaviour	Predictable	Tracks recent P99 × factor
Changing latency	Doesn't adapt	Self-tunes
Risk	False timeouts on slow days	Hides bugs: no timeouts, no alert
Use for	Strict SLAs, security paths	External APIs, ML inference

sdm w02 day-01-timeout-hell tradeoff W02 · Day 1 · Timeout Hell
Classic mistakes

No timeout, or 30 s "to be safe": threads pile up and you look dead.
Timeout = average or P99, failing healthy requests every day.
Adaptive timeouts without alerting on the timeout value and a hard maximum.

sdm w02 day-01-timeout-hell mistakes W02 · Day 1 · Timeout Hell
Your service calls 3 downstream services. How do you set timeouts? Start from the SLA (5 s) and each service's P99 and criticality. Budget 90% of the SLA, give each P99 × 2–3, parallelize independent calls, let optional ones fail open. sdm w02 day-01-timeout-hell ask W02 · Day 2 · Idempotency in Practice
Request lost or response lost? A client can't tell them apart: both look like a timeout. No timeout tuning fixes that; the only answer is making the operation safe to retry. sdm w02 day-02-idempotency-in-practice idea W02 · Day 2 · Idempotency in Practice
The idempotency key The client sends a unique key per logical operation. The server records it with status "processing" before calling the bank, stores the result, and returns the cached result for any repeat. sdm w02 day-02-idempotency-in-practice idea W02 · Day 2 · Idempotency in Practice
Pass the key downstream Send your idempotency key to the bank too: it deduplicates as well, and on an ambiguous timeout you can ask it what happened to that key, then update your record. sdm w02 day-02-idempotency-in-practice idea W02 · Day 2 · Idempotency in Practice
Guard the edge cases Hash the request body so a reused key with a different amount is rejected. Only completed requests block retries; a failed one may run again. Extend the TTL on completion. sdm w02 day-02-idempotency-in-practice idea W02 · Day 2 · Idempotency in Practice
Numbers to remember

≥ 10× the maximum retry window: minimum record TTL
24 hours practical default TTL (Stripe uses 24 h)
7+ days for financial transactions and compliance
255 maximum characters in a Stripe Idempotency-Key

sdm w02 day-02-idempotency-in-practice numbers W02 · Day 2 · Idempotency in Practice
Who generates the key?

	Client UUID	Server (payment intent)	Deterministic
Best for	Trusted clients	Public APIs, web	Internal services
Upside	No round-trip	Key quality	No state
Downside	Weak or lost keys	Extra round-trip	Blocks real duplicates

sdm w02 day-02-idempotency-in-practice tradeoff W02 · Day 2 · Idempotency in Practice
Classic mistakes

POST payments with no idempotency key: a double-click charges twice.
A TTL shorter than the retry window, so a late retry processes again.
Marking a partial failure "failed": a retry charges the customer twice.

sdm w02 day-02-idempotency-in-practice mistakes W02 · Day 2 · Idempotency in Practice
The first request timed out. Did the payment go through? Unknown, so the record says "processing". On a retry with the same key, ask the bank about that key, mark the record completed or failed, and return a consistent answer. sdm w02 day-02-idempotency-in-practice ask W02 · Day 3 · Circuit Breakers
Fail fast when it's clearly broken Timeouts and retries still wait 3.5 s per call and pile load onto a failing bank. A circuit breaker stops calling: users get an error in ~10 ms and the bank can recover. sdm w02 day-03-circuit-breakers idea W02 · Day 3 · Circuit Breakers
Closed, open, half-open Closed: calls pass and failures are counted. Open: calls fail immediately until a timer expires (often 30–60 s). Half-open: one test call decides, closed on success or open again. sdm w02 day-03-circuit-breakers idea W02 · Day 3 · Circuit Breakers
Count what's really a failure Timeouts, 5xx and refused connections count. 4xx errors and business rejections like "card declined" don't: the service worked correctly and said no. sdm w02 day-03-circuit-breakers idea W02 · Day 3 · Circuit Breakers
Use failure rate, not count "5 failures in 60 s" trips on Black Friday at 10,000 req/s even though 0.05% is normal. Open at a failure rate (around 50%) with a minimum number of calls. sdm w02 day-03-circuit-breakers idea W02 · Day 3 · Circuit Breakers
Retry, breaker and bulkhead together Check the circuit breaker first (cheapest), take a bulkhead slot second, retry transient failures last, with exponential backoff and jitter. They're layers, not alternatives. sdm w02 day-03-circuit-breakers idea W02 · Day 3 · Circuit Breakers
Numbers to remember

3,500 s vs 17.5 s slow-failure waiting in a 10-minute outage, without vs with a breaker
50% · 30% · 80% rate thresholds: normal, critical, best-effort
3 retries turn 100 requests into up to 400

sdm w02 day-03-circuit-breakers numbers W02 · Day 3 · Circuit Breakers
Local vs distributed breaker state

	Local state	Distributed (Redis)
Latency	Fastest	Adds a Redis round trip
Consistency	Each server decides alone	All servers agree
Dependency	None	Redis must be up

sdm w02 day-03-circuit-breakers tradeoff W02 · Day 3 · Circuit Breakers
Classic mistakes

Closing to full traffic at once: the backed-up herd crashes the recovering service. Ramp up.
Alerting only on error rate: an open circuit lowers it and hides the outage.
A fallback for payments, auth or inventory: you can't fake those.

sdm w02 day-03-circuit-breakers mistakes W02 · Day 3 · Circuit Breakers
What if the circuit opens during Black Friday? Use a failure rate, not a count, so 10x traffic doesn't trip it. Count real service failures, not declines. Keep a backup processor, and tell users to retry in 60 s. sdm w02 day-03-circuit-breakers ask W02 · Day 4 · Webhook Delivery System
At-least-once plus idempotent receivers True exactly-once is impossible (the Two Generals problem). Retry until acknowledged and give every event an ID the receiver uses as its idempotency key; a duplicate costs one lookup. sdm w02 day-04-webhook-delivery idea W02 · Day 4 · Webhook Delivery System
The pipeline Event sources publish to a durable bus (Kafka); a dispatcher fans each event out to its subscriptions; delivery queues partitioned by subscription feed workers; exhausted retries go to a dead letter queue. sdm w02 day-04-webhook-delivery idea W02 · Day 4 · Webhook Delivery System
Retry what can succeed Retry network errors, timeouts, 5xx and 429 (honour Retry-After). A 4xx means the request is malformed: retrying won't help, so log it or dead-letter it. sdm w02 day-04-webhook-delivery idea W02 · Day 4 · Webhook Delivery System
Sign and timestamp every webhook HMAC-SHA256 over the exact bytes sent, with a timestamp inside the signature. Receivers compare in constant time and reject anything more than 5 minutes old, which stops replays. sdm w02 day-04-webhook-delivery idea W02 · Day 4 · Webhook Delivery System
Recover fast after an outage Monitor endpoint health. When a dead endpoint recovers, move its pending retries to an immediate queue and ramp up gradually, within its rate limit, instead of waiting out the backoff. sdm w02 day-04-webhook-delivery idea W02 · Day 4 · Webhook Delivery System
Numbers to remember

~39 hours retry window: 1m, 5m, 15m, 1h, 2h, 4h, 8h, 24h
278/s 1M events/hour; ×3 subscriptions ≈ 833/s
13 min vs ~2 h to drain 10,000 events, with vs without health checks
5 min maximum timestamp age a receiver accepts

sdm w02 day-04-webhook-delivery numbers W02 · Day 4 · Webhook Delivery System
At-most-once vs at-least-once

	At-most-once	At-least-once
On a network blip	Lost forever	Retried, delivered
Duplicates	Never	Possible: receiver dedupes
Use for	Analytics, logs	Payments, orders

sdm w02 day-04-webhook-delivery tradeoff W02 · Day 4 · Webhook Delivery System
Classic mistakes

Re-serializing the payload after signing, so the receiver's HMAC over the raw body fails.
Retrying 4xx responses that will never succeed.
Delivering events for one entity in parallel: partition by entity to keep order.

sdm w02 day-04-webhook-delivery mistakes W02 · Day 4 · Webhook Delivery System
A receiver is down 2 hours. What happens to its 10,000 events? Spread across backoff stages, stragglers trickle in for ~2 hours. Health monitoring spots the recovery and moves them to an immediate queue; rate-limited delivery finishes in ~13 minutes. sdm w02 day-04-webhook-delivery ask W02 · Day 5 · Distributed Cron & Job Scheduling
Locks alone don't work A plain lock stays held when its server crashes. A lock timeout lets a slow job run twice. A Redis lock released with GET then DELETE can delete the next holder's lock. sdm w02 day-05-distributed-cron idea W02 · Day 5 · Distributed Cron & Job Scheduling
Perfect is impossible, so be practical FLP says consensus can't be guaranteed with even one faulty process. Production cron runs at-least-once, makes jobs idempotent, tolerates small delays and alerts on missed runs. sdm w02 day-05-distributed-cron idea W02 · Day 5 · Distributed Cron & Job Scheduling
One leader schedules Elect a leader with a lease in etcd, ZooKeeper or Consul; followers wait to take over. Use a coordination service instead of building your own unless you build databases. sdm w02 day-05-distributed-cron idea W02 · Day 5 · Distributed Cron & Job Scheduling
Fencing tokens stop stale leaders Each new leader gets a higher token and sends it with every operation; resources reject older tokens. A deposed leader paused by GC can't do damage when it wakes up. sdm w02 day-05-distributed-cron idea W02 · Day 5 · Distributed Cron & Job Scheduling
Track jobs outside the leader Record executions in a job store with heartbeats. A new leader skips jobs still beating, a stuck-job detector reschedules silent ones, and checkpoints let a job resume instead of redoing work. sdm w02 day-05-distributed-cron idea W02 · Day 5 · Distributed Cron & Job Scheduling
Numbers to remember

10 s etcd lease TTL in the lesson's leader-crash timeline
30 s job heartbeat interval in the payment job example
1 minute minimum Kubernetes CronJob granularity
1985 FLP impossibility (Fischer, Lynch, Paterson)

sdm w02 day-05-distributed-cron numbers W02 · Day 5 · Distributed Cron & Job Scheduling
Celery Beat vs Kubernetes CronJobs vs Temporal

	Celery Beat	K8s CronJobs	Temporal
Crash recovery	Manual	Automatic	Automatic
Exactly-once	No	No ("approximately once")	Workflow logic yes
Best for	Simple Python apps	Cloud-native apps	Critical workflows

sdm w02 day-05-distributed-cron tradeoff W02 · Day 5 · Distributed Cron & Job Scheduling
Classic mistakes

A lock with no expiry: one crash and the job never runs again.
Releasing a lock you may no longer own; check and delete atomically.
Non-idempotent jobs: a failover rerun sends the same 50 emails again.

sdm w02 day-05-distributed-cron mistakes W02 · Day 5 · Distributed Cron & Job Scheduling
The leader crashes mid-job. What happens? On another worker, the new leader sees a fresh heartbeat in the job store and doesn't reschedule. If it died too, the stuck-job detector reschedules it; idempotent, checkpointed work resumes safely. sdm w02 day-05-distributed-cron ask W02 · Boss · The Ultimate System Design Interview
The spike is the problem 5 million users hit one 10-minute sale and 80% of traffic lands in the first 30 seconds: ~150,000 requests a second at peak. CDN, WAF and rate limits absorb the herd before it reaches services. sdm w02 capstone-interview-week-2-flash-sale-for-megamart idea W02 · Boss · The Ultimate System Design Interview
Atomic claims in Redis A Lua script checks the user's existing claim and decrements the inventory counter in one atomic step, restoring it if it goes below zero. No database in the hot path, so no overselling. sdm w02 capstone-interview-week-2-flash-sale-for-megamart idea W02 · Boss · The Ultimate System Design Interview
Claims expire back into the pool A claim holds a unit for 3 minutes. A claim-expiry worker returns unpaid units to inventory, triggering a second wave of claims (~15,000 claim operations for 10,000 units). sdm w02 capstone-interview-week-2-flash-sale-for-megamart idea W02 · Boss · The Ultimate System Design Interview
Checkout uses every Week 2 pattern Validate the claim, check the idempotency store by claim_id, call Stripe behind a circuit breaker with a timeout and its idempotency key, create the order in a transaction, publish events asynchronously. sdm w02 capstone-interview-week-2-flash-sale-for-megamart idea W02 · Boss · The Ultimate System Design Interview
Precompute notifications Build the 2.4M-recipient list 10 minutes before the sale into Redis batches. At start, 24 workers fan out at once and queue everything in ~10 seconds; push and email have separate pools. sdm w02 capstone-interview-week-2-flash-sale-for-megamart idea W02 · Boss · The Ultimate System Design Interview
Numbers to remember

150,000 req/s design peak: (5M × 0.8) ÷ 30 s, rounded up
333 claims/s 10,000 units claimed in the first 30 s
80,000/s notifications: 2.4M within 30 seconds
10 per minute claim attempts allowed per user

sdm w02 capstone-interview-week-2-flash-sale-for-megamart numbers W02 · Boss · The Ultimate System Design Interview
Claim design alternatives

	Chosen	Alternative	Why not
Claims	Direct Redis decrement	Queue every claim	Too slow for "instant"
Inventory	Decrement on claim	Decrement at checkout	Shows stock that's gone
Locking	One atomic counter	Lock per unit	10,000 locks

sdm w02 capstone-interview-week-2-flash-sale-for-megamart tradeoff W02 · Boss · The Ultimate System Design Interview
Classic mistakes

Checking stock then decrementing in two steps: a race that oversells.
Computing millions of notification recipients at the moment the sale starts.
Putting the database in the claim path at 40,000 claim attempts a second.

sdm w02 capstone-interview-week-2-flash-sale-for-megamart mistakes W02 · Boss · The Ultimate System Design Interview
What changes at 10x scale? Shard inventory by region with eventual consistency for the display count and strong consistency only for claims, edge-cache sale pages, use WebSockets instead of polling, and scale the database with Vitess or CockroachDB. sdm w02 capstone-interview-week-2-flash-sale-for-megamart ask W03 · Day 1 · Queue vs Stream
Work to do vs things that happened A queue holds jobs: each message goes to one worker and is deleted after the ack. A stream is an append-only log: events stay, each reader tracks its own offset and can replay. sdm w03 day-01-queue-vs-stream idea W03 · Day 1 · Queue vs Stream
Consumer groups Kafka's model: every group gets all messages (fan-out) and within a group each partition goes to exactly one consumer (load balancing). Parallelism equals the partition count. sdm w03 day-01-queue-vs-stream idea W03 · Day 1 · Queue vs Stream
Order lives in the partition Kafka orders messages only within a partition. Key by the entity that needs order (order_id, user_id) so "created, paid, shipped" can't be processed out of order. sdm w03 day-01-queue-vs-stream idea W03 · Day 1 · Queue vs Stream
At-least-once plus idempotent consumers Exactly-once needs special support on both sides, so most systems ack after processing and dedupe, e.g. by partition and offset stored in the same transaction as the work. sdm w03 day-01-queue-vs-stream idea W03 · Day 1 · Queue vs Stream
Numbers to remember

6–12 partitions for a new topic; 50–100+ for high-volume ones
9,000/s lag growth when producing 10,000/s and consuming 1,000/s: 32.4M behind in an hour
7+ trillion messages a day at LinkedIn, where Kafka was born
`acks=all` producer setting for durability

sdm w03 day-01-queue-vs-stream numbers W03 · Day 1 · Queue vs Stream
Queue vs stream

	Queue (RabbitMQ, SQS)	Stream (Kafka, Kinesis)
After reading	Deleted	Retained
Consumers per message	One	Every group
Replay	No	Yes, rewind the offset
Use for	Jobs, tasks, webhooks	Event sourcing, analytics, fan-out

sdm w03 day-01-queue-vs-stream tradeoff W03 · Day 1 · Queue vs Stream
Classic mistakes

Kafka for simple background jobs that a queue with built-in retry would handle.
No partition key, so one order's events land on different partitions.
One partition "to start": only one consumer can ever work on it.

sdm w03 day-01-queue-vs-stream mistakes W03 · Day 1 · Queue vs Stream
Queue or stream when email, warehouse and analytics all need order events? A stream: several consumer groups each need every event, and replay helps recovery. Partition by user_id for per-user order, and make each consumer idempotent. sdm w03 day-01-queue-vs-stream ask W03 · Day 2 · Transactional Outbox
The dual-write problem Saving an order then publishing to Kafka are two writes with no shared transaction. A crash in between leaves an order no downstream service hears about; flipping them publishes events for orders that never saved. sdm w03 day-02-transactional-outbox idea W03 · Day 2 · Transactional Outbox
Write the event to the database Insert the business row and an outbox row in the same transaction: both commit or neither does. A separate publisher reads unpublished outbox rows and sends them to Kafka. sdm w03 day-02-transactional-outbox idea W03 · Day 2 · Transactional Outbox
Polling or CDC A poller queries `published_at IS NULL` every few seconds: simple, but adds latency and database load. CDC (Debezium) tails the write-ahead log: ~10–100 ms latency, no polling, more to operate. sdm w03 day-02-transactional-outbox idea W03 · Day 2 · Transactional Outbox
Still at-least-once A publisher that crashes after Kafka acks but before marking the row published sends it again. Consumers dedupe by event ID; Kafka transactions can't cover the database update. sdm w03 day-02-transactional-outbox idea W03 · Day 2 · Transactional Outbox
Numbers to remember

~10–100 ms Debezium CDC from INSERT to Kafka
> 10K events/s or < 1 s latency: move from polling to CDC
10,000 · 5 min alert thresholds: unpublished rows, oldest unpublished age
7 days keep published rows, then delete them

sdm w03 day-02-transactional-outbox numbers W03 · Day 2 · Transactional Outbox
Polling publisher vs CDC

	Polling	CDC (Debezium)
Latency	Up to the poll interval	Milliseconds
Database load	Extra queries	Reads the WAL
Setup	Any database, no extra parts	WAL access, Kafka Connect
Start with	Low volume	High volume, low latency

sdm w03 day-02-transactional-outbox tradeoff W03 · Day 2 · Transactional Outbox
Classic mistakes

Writing the outbox row in a separate transaction: the dual write is back.
Publishing to Kafka inside the request path, so Kafka trouble fails user requests.
No alert on outbox lag: the table grows to 10M rows before anyone notices.

sdm w03 day-02-transactional-outbox mistakes W03 · Day 2 · Transactional Outbox
What if Kafka is down for an hour? Events wait in the outbox, so nothing is lost and user requests still succeed. When Kafka recovers the publisher catches up; scale it out temporarily to drain the backlog. sdm w03 day-02-transactional-outbox ask W03 · Day 3 · Backpressure and Flow Control
Async removes natural backpressure A synchronous client can't send faster than the server answers. A queue absorbs the gap instead, until memory runs out: absorption, then resource exhaustion, then collapse. sdm w03 day-03-backpressure-and-flow-control idea W03 · Day 3 · Backpressure and Flow Control
Watch lag, depth and latency Consumer lag (alert at 10,000 messages or 5 minutes), queue depth (100,000 or growing for 5 minutes) and queue wait time. Throughput falling while lag grows means consumers are struggling. sdm w03 day-03-backpressure-and-flow-control idea W03 · Day 3 · Backpressure and Flow Control
Four responses Shed low-priority load (metrics, logs), rate-limit producers (adaptively, as lag grows), scale consumers (Kafka: up to the partition count), or spill overflow to disk or S3 for later. sdm w03 day-03-backpressure-and-flow-control idea W03 · Day 3 · Backpressure and Flow Control
You get two of three Completeness, latency, stability: pick two. Most systems keep completeness and stability and accept higher latency during spikes; real-time pricing drops stale events instead. sdm w03 day-03-backpressure-and-flow-control idea W03 · Day 3 · Backpressure and Flow Control
Recover gradually After an outage, don't jump from 100 to 10,000 messages a second: the backlog floods downstream. Step up (500, 1,000, 2,500, 5,000, 10,000) after a minute of stability each. sdm w03 day-03-backpressure-and-flow-control idea W03 · Day 3 · Backpressure and Flow Control
Numbers to remember

5,000 vs 200,000/s design rate vs the spike that took a site down
12 partitions at most 12 Kafka consumers
~3.5 hours 10M bulk emails at 800/s
30 s age at which surge-pricing events are dropped

sdm w03 day-03-backpressure-and-flow-control numbers W03 · Day 3 · Backpressure and Flow Control
Backpressure strategies

	Data loss	Latency	Best for
Load shedding	Yes, controlled	None	Metrics, logs
Rate limiting	No	Producers wait	API traffic
Scaling	No	Minimal	Sustained load
Spillover	No	Higher for overflow	Must-keep messages

sdm w03 day-03-backpressure-and-flow-control tradeoff W03 · Day 3 · Backpressure and Flow Control
Classic mistakes

Unbounded in-memory queues: the OOM killer takes every message with it.
Rate limiting only at the API gateway while internal services overwhelm the database.
Hot partition keys: one consumer drowns while the others idle.

sdm w03 day-03-backpressure-and-flow-control mistakes W03 · Day 3 · Backpressure and Flow Control
A 10M-email campaign hits a 1,000/s provider cap. How do password resets get through? Separate priority queues: critical first with a guaranteed 200/s, bulk up to 800/s when critical is empty. The campaign drains in ~3.5 hours. sdm w03 day-03-backpressure-and-flow-control ask W03 · Day 4 · Dead Letters and Poison Pills
A poison pill blocks everything behind it A message that crashes the consumer every time causes a crash loop: restart, fetch the same message, crash. In the lesson, one bad message held 50,000 orders for 4 hours. sdm w03 day-04-dead-letters-and-poison-pills idea W03 · Day 4 · Dead Letters and Poison Pills
Classify the failure Transient (timeouts, rate limits): retry with backoff. Permanent (bad JSON, missing field, rule violation): straight to the DLQ. Poison pills (crashes, hangs, memory blowups): quarantine immediately. sdm w03 day-04-dead-letters-and-poison-pills idea W03 · Day 4 · Dead Letters and Poison Pills
Delayed retry topics Instead of retrying in place, republish to retry topics with growing delays (1 s, 10 s, 60 s), then the DLQ. The main topic keeps flowing; Uber described this design for Kafka. sdm w03 day-04-dead-letters-and-poison-pills idea W03 · Day 4 · Dead Letters and Poison Pills
A DLQ is an operational tool Store the original message, error and stack trace, retry count, timestamps and source topic, partition and offset. Monitor depth, growth and age, triage daily, and build replay tooling from day one. sdm w03 day-04-dead-letters-and-poison-pills idea W03 · Day 4 · Dead Letters and Poison Pills
Replay needs idempotency Check the payment_id before reprocessing so a replay can't double-charge. Old messages may not match today's schema: version them, migrate on replay, or close them out. sdm w03 day-04-dead-letters-and-poison-pills idea W03 · Day 4 · Dead Letters and Poison Pills
Numbers to remember

3–5 retries with exponential backoff: the default
0 retries for validation errors
2x normal processing time as the poison-pill timeout
7 · 90 days hot DLQ retention, then cold archive

sdm w03 day-04-dead-letters-and-poison-pills numbers W03 · Day 4 · Dead Letters and Poison Pills
How many retries?

	Too few (1–2)	Too many (10+)
Transient errors	Land in the DLQ needlessly	Recover
Poison pills	Quarantined fast	Block the queue longer
Cost	More manual replays	Wasted retries, slower detection

sdm w03 day-04-dead-letters-and-poison-pills tradeoff W03 · Day 4 · Dead Letters and Poison Pills
Classic mistakes

Resetting the retry count to 0 in each consumer instead of reading it from headers: infinite loop.
A DLQ nobody monitors: it becomes a graveyard of lost data.
A DLQ consumer that processes instead of just storing and alerting.

sdm w03 day-04-dead-letters-and-poison-pills mistakes W03 · Day 4 · Dead Letters and Poison Pills
How do you stop a poison pill from freezing the pipeline? Time out processing and track repeated crashes on the same message; quarantine it in the DLQ at once instead of retrying. Only transient errors get retries. sdm w03 day-04-dead-letters-and-poison-pills ask W03 · Day 5 · Audit Log System
Who did what, to which resource, when, from where An audit log serves auditors, legal and security, not engineers: strict schema, retention in years, and it must be immutable. An application log is for debugging and can be deleted. sdm w03 day-05-audit-log-system idea W03 · Day 5 · Audit Log System
Capture in the same transaction Write the audit event to an outbox in the same transaction as the business action, so a transfer can't succeed without its record. Kafka, consumers and the DLQ carry it asynchronously. sdm w03 day-05-audit-log-system idea W03 · Day 5 · Audit Log System
Immutable and tamper-evident No UPDATE or DELETE: corrections are new events. Each event carries the previous event's hash, checkpointed somewhere an attacker can't change, and cold files sit under S3 Object Lock. sdm w03 day-05-audit-log-system idea W03 · Day 5 · Audit Log System
Tiered storage Hot (0–90 days) in TimescaleDB for fast investigation queries, warm (to 1 year) in ClickHouse or BigQuery, cold (1–7 years) as Parquet on S3, queryable with Athena. sdm w03 day-05-audit-log-system idea W03 · Day 5 · Audit Log System
Never shed audit events When consumers fall behind, scale them, batch harder and alert, but accept latency: audit data can't be dropped. Sort investigations by ingestion time and keep both timestamps. sdm w03 day-05-audit-log-system idea W03 · Day 5 · Audit Log System
Numbers to remember

50,000/s peak audit events to design for (5,000 normally)
7 years SOX audit records and workpapers (SEC Rule 2-06), the usual retention target
90 days · 1 year hot, then warm storage boundaries
< 100 ms clock drift target, kept by NTP

sdm w03 day-05-audit-log-system numbers W03 · Day 5 · Audit Log System
Synchronous vs asynchronous audit

	Synchronous	Asynchronous (outbox)
User latency	+50–200 ms	+0 ms
Visibility	Immediate	1–60 s
Risk	Audit DB down, app down	Audit lags during issues

sdm w03 day-05-audit-log-system tradeoff W03 · Day 5 · Audit Log System
Classic mistakes

Writing the audit event after the business transaction: money moves with no trail.
"Fixing" audit rows with UPDATE, which breaks the chain of custody.
Logging only "delete doc_123" without actor, context or reason.

sdm w03 day-05-audit-log-system mistakes W03 · Day 5 · Audit Log System
How do you prove the audit logs weren't tampered with? Hash-chain events so editing one means recomputing every later hash, and checkpoint the latest hash to WORM storage or a signed log. Keep cold files under S3 Object Lock. sdm w03 day-05-audit-log-system ask W03 · Boss · Event-Driven Food Delivery Order System
Size the dinner rush 500,000 orders a day, 60% in 6–9 PM: ~28 orders/s, ~56 at the peak, design for 100. With ~8 events per order and 7 consumer services, that's 5,600 event deliveries a second. sdm w03 capstone-interview-week-3-event-driven-order-system idea W03 · Boss · Event-Driven Food Delivery Order System
Kafka keyed by order_id Seven services need the same events, replay helps debugging, and each order's events must stay in order. Partition `orders.events` (32 partitions) by order_id; each service is its own consumer group. sdm w03 capstone-interview-week-3-event-driven-order-system idea W03 · Boss · Event-Driven Food Delivery Order System
Different consumers, different guarantees Payment needs effectively-once with an idempotency key per order (Stripe won't charge twice). Analytics can lag, push is fire-and-forget, fraud detection needs real time. sdm w03 capstone-interview-week-3-event-driven-order-system idea W03 · Boss · Event-Driven Food Delivery Order System
Retry topics, then the DLQ Transient failures go to retry topics delayed 1, 5 and 15 minutes; after 3 retries, the DLQ. Payment failures retry then notify the customer; restaurant rejections find an alternative or refund. sdm w03 capstone-interview-week-3-event-driven-order-system idea W03 · Boss · Event-Driven Food Delivery Order System
Audit for disputes 2% of orders are disputed. Every service writes immutable audit events, so support queries one order's full timeline in minutes instead of correlating logs across services. sdm w03 capstone-interview-week-3-event-driven-order-system idea W03 · Boss · Event-Driven Food Delivery Order System
Numbers to remember

100 orders/s design target (peak ~56)
5,600/s event deliveries: 800 events/s × 7 consumers
10K · 50K · 100K consumer lag: warning, critical, emergency
2 years audit retention: ~5.8 TB of audit data

sdm w03 capstone-interview-week-3-event-driven-order-system numbers W03 · Boss · Event-Driven Food Delivery Order System
Responses by pressure level

	Warning (10K lag)	Critical (50K)	Emergency (100K)
Scale	Pre-warm consumers	Auto-scale	Max replicas
Shed	Nothing	Low priority	Normal and low
People	Slack alert	Page on-call	Rate-limit the API

sdm w03 capstone-interview-week-3-event-driven-order-system tradeoff W03 · Boss · Event-Driven Food Delivery Order System
Classic mistakes

Saving the order and publishing to Kafka as two writes: lost or phantom orders.
Shedding driver-assignment events: a late driver hurts more than late analytics.
Retrying forever in the main topic, so one failing order blocks the rest.

sdm w03 capstone-interview-week-3-event-driven-order-system mistakes W03 · Boss · Event-Driven Food Delivery Order System
Kafka is down 30 minutes at dinner. What happens? Orders are still accepted: events pile up in the outbox while the database is healthy. A lag alert fires within a minute; when Kafka returns, the publisher catches up. sdm w03 capstone-interview-week-3-event-driven-order-system ask W04 · Day 1 · Caching Patterns
Why cache Latency (database 10–100 ms, Redis 0.5–2 ms, local memory 0.0001 ms), throughput (5,000 database queries/s vs 100,000+ Redis ops/s) and cost. "Just add Redis" isn't a strategy. sdm w04 day-01-caching-patterns idea W04 · Day 1 · Caching Patterns
Cache-aside is the default The app checks the cache, on a miss reads the database and fills the cache, and on writes invalidates the key. Only used data gets cached, and a cache outage just makes things slower. sdm w04 day-01-caching-patterns idea W04 · Day 1 · Caching Patterns
Write-through vs write-behind Write-through updates cache and database together: fresh reads, slower writes, for balances and inventory. Write-behind acks after the cache and flushes later in batches: fastest, but a cache crash loses data. sdm w04 day-01-caching-patterns idea W04 · Day 1 · Caching Patterns
Mix patterns per data type In one store: catalog cache-aside, inventory write-through (or no cache during a sale), prices cache-aside with event-driven invalidation, view counts write-behind. Checkout always reads the database. sdm w04 day-01-caching-patterns idea W04 · Day 1 · Caching Patterns
TTL is the safety net Invalidation can fail on a Redis timeout. A TTL sized to the staleness you can accept means a stale value still expires: product details 1 hour, prices 5 minutes, inventory 30 seconds. sdm w04 day-01-caching-patterns idea W04 · Day 1 · Caching Patterns
Numbers to remember

10,000 × 5 = 50,000 queries/s on a database built for 5,000: the outage
99% → 0% hit ratio after a restart: 10 → 1,000 req/s reach the database
> 95% hit ratio target; alert below 80–90%
80% of maxmemory: alert before evictions spike

sdm w04 day-01-caching-patterns numbers W04 · Day 1 · Caching Patterns
The four caching patterns

	Write latency	Consistency	Data loss risk
Cache-aside	Database	Eventual	Low
Read-through	Database	Eventual	Low
Write-through	Cache + database	Fresh after writes	Low
Write-behind	Cache only	Eventual	High

sdm w04 day-01-caching-patterns tradeoff W04 · Day 1 · Caching Patterns
Classic mistakes

Letting a cache error become a 500 instead of falling back to the database.
Caching everything: rarely read data evicts the hot set and lowers the hit ratio.
Collision-prone keys like `user`; use `product:{id}:v1` with a schema version.

sdm w04 day-01-caching-patterns mistakes W04 · Day 1 · Caching Patterns
Flash-sale prices change instantly. How does the cache keep up? Event-driven invalidation: update the database, publish an event, and a service deletes the affected keys, typically within a second. Keep a 5-minute TTL in case an event is lost. sdm w04 day-01-caching-patterns ask W04 · Day 2 · Invalidation Strategies
The cache doesn't know the data changed A flash sale moved a price from $99 to $79, but the 1-hour TTL kept showing $99 and 147 customers complained. Match each data type's invalidation to its staleness tolerance. sdm w04 day-02-cache-invalidation idea W04 · Day 2 · Invalidation Strategies
TTL, events, versions TTL: simple, stale until expiry. Event-driven: publish on change and delete the key in ~100–500 ms, but needs Kafka and a DLQ. Versioned keys: bump `product:123:v5` to `v6`, or a global version to invalidate everything. sdm w04 day-02-cache-invalidation idea W04 · Day 2 · Invalidation Strategies
Delete, don't update Write-update races: two writers can leave the cache holding the older value. Deleting is idempotent and the next read loads fresh data; update in place only with one writer or atomic ops like INCR. sdm w04 day-02-cache-invalidation idea W04 · Day 2 · Invalidation Strategies
Invalidate after commit Deleting inside the transaction lets a reader cache the old row before commit, leaving cache and database apart indefinitely. Facebook's leases go further: a fill after an invalidation is refused. sdm w04 day-02-cache-invalidation idea W04 · Day 2 · Invalidation Strategies
Decompose the cache Cache `product:123:details` (24 h), `price` (5 min plus events), `inventory` (30 s plus events) and `reviews` (1 h) separately, so an inventory change doesn't evict the description. sdm w04 day-02-cache-invalidation idea W04 · Day 2 · Invalidation Strategies
Numbers to remember

TTL = staleness × 0.8 e.g. 5 minutes tolerated → 4-minute TTL
~100–500 ms event-driven invalidation latency
99% · 1% invalidations handled by events vs by the TTL safety net
< 30 s · < 5 min staleness tolerance: inventory, prices

sdm w04 day-02-cache-invalidation numbers W04 · Day 2 · Invalidation Strategies
Invalidation strategies

	Freshness	Complexity	Use for
TTL only	Minutes	Low	Static, tolerant reads
Event-driven	Seconds	High	Prices, inventory
Versioned keys	Seconds	Medium	Bulk updates, deploys
TTL + events	Seconds	Medium-high	Most systems

sdm w04 day-02-cache-invalidation tradeoff W04 · Day 2 · Invalidation Strategies
Classic mistakes

Invalidating only `product:123` and forgetting category pages, search results and deals.
Pattern deletes (`KEYS category:*`) in the hot path; track keys per category in a set.
Event-driven invalidation with no TTL: one lost event means stale forever.

sdm w04 day-02-cache-invalidation mistakes W04 · Day 2 · Invalidation Strategies
You need to invalidate millions of keys at once. How? Put a global version in the key: increment one counter and every old key misses, then expires through its TTL. No mass delete needed; keep it as the emergency option. sdm w04 day-02-cache-invalidation ask W04 · Day 3 · Thundering Herd
One expiring key can take the site down When a hot key expires, every concurrent request misses and queries the database: 1 miss becomes N queries. 50,000 homepage requests met a database built for 5,000 queries a second. sdm w04 day-03-thundering-herd idea W04 · Day 3 · Thundering Herd
Coalesce misses into one fetch Request coalescing (single flight) lets concurrent requests for a missing key join one in-flight fetch and share its result. Instagram caches a promise for the value; locking does the same with more waiting. sdm w04 day-03-thundering-herd idea W04 · Day 3 · Thundering Herd
Never let hot keys expire Background refresh recomputes known hot keys (homepage, trending, deals) before their TTL, so users never miss. For a scheduled change, pre-warm the new key and switch atomically. sdm w04 day-03-thundering-herd idea W04 · Day 3 · Thundering Herd
Serve stale while revalidating When an entry goes stale, return it immediately and refresh in the background. If the refresh fails, extend the stale window instead of letting it lapse onto a struggling database. sdm w04 day-03-thundering-herd idea W04 · Day 3 · Thundering Herd
Desynchronize expiry Jitter TTLs and refresh probabilistically just before expiry, weighted by how long recomputing takes, so keys loaded together don't all expire together. sdm w04 day-03-thundering-herd idea W04 · Day 3 · Thundering Herd
Numbers to remember

10 s · 5 s lock TTL; how long a waiter waits before fetching itself
> 10K/s per key prefer coalescing; locking for moderate traffic
90 s TTL · 60 s refresh trending cache: 30 s of buffer

sdm w04 day-03-thundering-herd numbers W04 · Day 3 · Thundering Herd
Thundering herd defenses

	Latency cost	Complexity	Best for
Locking	High: waiters block	Low	Moderate traffic
Coalescing	Low	Medium	High traffic
Probabilistic expiry	None	Low	Many keys, distributed
Background refresh	None	High	Known hot keys
Stale-while-revalidate	None	Medium	Stale is fine

sdm w04 day-03-thundering-herd tradeoff W04 · Day 3 · Thundering Herd
Classic mistakes

One TTL for everything, so the homepage expiring sends 10K queries at once.
Coalesced waiters with no timeout; fall back to stale data instead.
Deploying with a cold cache; warm hot keys behind a health-check gate first.

sdm w04 day-03-thundering-herd mistakes W04 · Day 3 · Thundering Herd
100,000 req/s for one trending key. How do you cache it? Background refresh every 60 s with a 90 s TTL, stale-while-revalidate if the job fails, Redis replicas for reads, and a 5 s local cache per app server. sdm w04 day-03-thundering-herd ask W04 · Day 4 · Feed Caching
Reads vs writes pull apart Computing a feed on read means querying every followee (500 × 10 ms = 5 s). Precomputing it means every post fans out to every follower: one celebrity post is 50 million cache writes. sdm w04 day-04-feed-caching idea W04 · Day 4 · Feed Caching
Fan-out on write When a user posts, push the post ID into each follower's feed cache (a Redis sorted set scored by time). Reads become one cache lookup; the cost moves to writes and storage. sdm w04 day-04-feed-caching idea W04 · Day 4 · Feed Caching
Hybrid push and pull Push posts from regular users (under ~10K followers); store celebrity posts once and pull them at read time, merging with the precomputed feed. Twitter and Instagram run mostly push. sdm w04 day-04-feed-caching idea W04 · Day 4 · Feed Caching
Cache IDs, trim, skip the dormant Store post IDs, not posts, and trim each feed (~800 entries). Skip long-inactive followers during fan-out and rebuild their feed when they return; fetch pulled posts in parallel. sdm w04 day-04-feed-caching idea W04 · Day 4 · Feed Caching
Numbers to remember

100M × 50 KB = 5 TB caching full feeds instead of IDs
~1.1M/s feed insertions: 500M tweets/day × 200 followers
50 min to push 300M writes at 100K/s: why celebrities are pulled
< 200 ms feed latency target

sdm w04 day-04-feed-caching numbers W04 · Day 4 · Feed Caching
Fan-out on write vs on read

	On write (push)	On read (pull)
Read latency	One cache read	N queries
Write cost	N cache writes	One write
Storage	A feed per user	Posts only
Best for	Regular users	Celebrities, 2nd-degree feeds

sdm w04 day-04-feed-caching tradeoff W04 · Day 4 · Feed Caching
Classic mistakes

Pure push for everyone: one celebrity post backs up the fan-out for an hour.
Feeds never trimmed, so sorted sets grow forever.
Fetching each followee's posts sequentially instead of in parallel.

sdm w04 day-04-feed-caching mistakes W04 · Day 4 · Feed Caching
How does LinkedIn's feed differ from Twitter's? LinkedIn shows 2nd-degree content ("John liked this"), so fanning out on every reaction multiplies writes. It computes on read: fetch connections' posts and reactions, dedupe, attribute and rank. sdm w04 day-04-feed-caching ask W04 · Day 5 · Multi-Tier Caching
Five layers, five jobs Browser (static assets, preferences), CDN (public content, media), API gateway (auth validation, rate limits, dedup), Redis (business data, sessions, flags), database buffer pool (automatic). sdm w04 day-05-multi-tier-caching idea W04 · Day 5 · Multi-Tier Caching
Invalidate closest to the database first Delete the Redis key, then the gateway cache, then purge the CDN asynchronously. Purge the CDN first and it may refetch the old value from a still-stale origin cache. Browsers expire only by TTL. sdm w04 day-05-multi-tier-caching idea W04 · Day 5 · Multi-Tier Caching
Never let the CDN cache personal data Mark authenticated responses `Cache-Control: private`. Vary on headers that change the content (Accept-Language, Accept-Encoding), never on Cookie: that gives every user their own cache entry. sdm w04 day-05-multi-tier-caching idea W04 · Day 5 · Multi-Tier Caching
Bounded staleness, not perfect consistency Design a maximum staleness per tier: CDN 60 s, Redis 5 minutes (usually seconds with events), browser 60 s for API data. Checkout bypasses every cache. sdm w04 day-05-multi-tier-caching idea W04 · Day 5 · Multi-Tier Caching
Numbers to remember

20 · 5 · 2 · 50 ms CDN, gateway, Redis, disk; browser 0
1 year, immutable hashed JS/CSS; HTML gets no-cache with an ETag
> 95% CDN hit ratio target for static assets
100% of Netflix's video traffic is served by Open Connect

sdm w04 day-05-multi-tier-caching numbers W04 · Day 5 · Multi-Tier Caching
Anonymous vs authenticated requests

	Anonymous	Authenticated
CDN	Cache aggressively	Don't cache personal data
Gateway	Cache API responses	Auth checks, rate limits
App cache	Shared data	Per-user data

sdm w04 day-05-multi-tier-caching tradeoff W04 · Day 5 · Multi-Tier Caching
Classic mistakes

An authenticated `/api/profile` cached at the CDN: one user's data served to another.
Missing `Vary: Accept-Language`, so Spanish users get the English page.
Unversioned keys: a deploy reads the old schema; put `v3` in the key.

sdm w04 day-05-multi-tier-caching mistakes W04 · Day 5 · Multi-Tier Caching
A price changes from $99 to $79. Walk through every tier. Update the database, delete the Redis key synchronously, queue the CDN purge (cache tags for bulk changes), let browsers expire within their 60 s TTL. Checkout reads the database. sdm w04 day-05-multi-tier-caching ask W04 · Boss · Design a Global E-Commerce Product Catalog System
At 600K req/s, the hit rate is the design 500M daily views is ~6,000 req/s, and flash sales spike it 100x. Even 1% misses at 600,000 req/s means 6,000 database queries a second: hit rate must exceed 99%. sdm w04 capstone-interview-week-4-ecommerce-product-catalog idea W04 · Boss · Design a Global E-Commerce Product Catalog System
Freshness per data type Metadata tolerates an hour (TTL), prices under 5 seconds (events plus a 30 s TTL), inventory under 30 seconds for display, and checkout reads live data. Images can be a week old. sdm w04 capstone-interview-week-4-ecommerce-product-catalog idea W04 · Boss · Design a Global E-Commerce Product Catalog System
Flash sale protection Pre-warm the deal before midnight, coalesce concurrent misses into one fetch, refresh hot keys in the background and serve stale while revalidating. Last Black Friday a single expiring key took the site down. sdm w04 capstone-interview-week-4-ecommerce-product-catalog idea W04 · Boss · Design a Global E-Commerce Product Catalog System
Personalization by activity tier Precompute recently viewed and recommendations for active users, give recent users shorter TTLs, compute on demand for dormant ones. Caching all 100M users' 3.5 KB would waste most of 350 GB. sdm w04 capstone-interview-week-4-ecommerce-product-catalog idea W04 · Boss · Design a Global E-Commerce Product Catalog System
Degrade when Redis fails A circuit breaker detects the failure in under a second; serve stale data from a local in-process cache, hide personalization, pause flash-sale purchases to avoid overselling, then recover half-open after 30 seconds. sdm w04 capstone-interview-week-4-ecommerce-product-catalog idea W04 · Boss · Design a Global E-Commerce Product Catalog System
Numbers to remember

~300 GB Redis: 80 products, 200 personalization, 10 categories
< 50 ms · < 200 ms page latency on a cache hit vs a miss
10% of all traffic goes to Deal of the Day

sdm w04 capstone-interview-week-4-ecommerce-product-catalog numbers W04 · Boss · Design a Global E-Commerce Product Catalog System
Where each piece is cached

	Browser	CDN	Redis
Static assets	1 year	1 year	No
Anonymous page	60 s, swr 30 s	60 s	5 min, events
Price, inventory	No	No	30 s, events
Checkout	No	No	No

sdm w04 capstone-interview-week-4-ecommerce-product-catalog tradeoff W04 · Boss · Design a Global E-Commerce Product Catalog System
Classic mistakes

Letting the Deal of the Day key expire at midnight instead of pre-warming it.
Caching prices at the CDN: checkout then disagrees with the page.
Precomputing personalization for every registered user, active or not.

sdm w04 capstone-interview-week-4-ecommerce-product-catalog mistakes W04 · Boss · Design a Global E-Commerce Product Catalog System
What breaks first at 10x load? The database: a 1% miss rate at 6M req/s is 60,000 queries a second. Add read replicas and PgBouncer, or move product reads to DynamoDB. Redis grows to 30+ nodes; the CDN scales itself. sdm w04 capstone-interview-week-4-ecommerce-product-catalog ask W05 · Day 1 · Consistency Models
Who sees a write, and when? A write lands on Node 1 and a read hits Node 2 before replication arrives. Consistency models are precise answers to what that read may return: new value, old value, or an error. sdm w05 day-01-consistency-models idea W05 · Day 1 · Consistency Models
The spectrum, strongest first Linearizable (behaves like one copy), sequential (one agreed order), causal (cause before effect), read-your-writes, monotonic reads (never go backwards), eventual (converges, no promise when). sdm w05 day-01-consistency-models idea W05 · Day 1 · Consistency Models
Session guarantees users notice Read-your-writes: route a user's reads to the primary after their write, or track the write version. Monotonic reads: remember the last version read and only accept replicas at or past it. sdm w05 day-01-consistency-models idea W05 · Day 1 · Consistency Models
Choose per operation Browse products eventually consistent, reserve inventory and check out strongly consistent, order history read-your-writes. DynamoDB and Cassandra let you choose per read: eventual, QUORUM, ALL. sdm w05 day-01-consistency-models idea W05 · Day 1 · Consistency Models
Spanner buys strong consistency with time TrueTime (GPS and atomic clocks, uncertainty typically under 7 ms) plus Paxos and commit-wait give global linearizability, paid for in write latency: commit-wait plus a Paxos round trip, and more across regions. sdm w05 day-01-consistency-models idea W05 · Day 1 · Consistency Models
Numbers to remember

W + R > N reads overlap the latest write: N=3, W=2, R=2
50% cheaper: DynamoDB eventually consistent reads
< 7 ms Spanner's typical TrueTime uncertainty
48 hours DNS propagation: eventual consistency you know

sdm w05 day-01-consistency-models numbers W05 · Day 1 · Consistency Models
CP vs AP during a partition

	CP	AP
Behaviour	Minority side becomes unavailable	Every node keeps serving
Data	Never wrong	May diverge, merged later
Examples	ZooKeeper, etcd, Spanner	Cassandra, DNS, caches
Use for	Money, inventory, leaders	Carts, sessions, counters

sdm w05 day-01-consistency-models tradeoff W05 · Day 1 · Consistency Models
Classic mistakes

Reading from a replica right after writing to the primary: the user's change "disappears".
Random replica per read, so a counter appears to go backwards.
Strong consistency for everything: latency and availability pay for reads that don't need it.

sdm w05 day-01-consistency-models mistakes W05 · Day 1 · Consistency Models
What consistency model for a social media feed? Causal: a comment must never appear before its post, but concurrent posts from different people can show in any order. It keeps the experience right without paying for strong consistency. sdm w05 day-01-consistency-models ask W05 · Day 2 · Distributed Transactions — The Saga Pattern
Why not two-phase commit 2PC holds locks from PREPARE until the decision, blocks if the coordinator dies, needs every service up at once, and most NoSQL stores, brokers like Kafka and third-party APIs (Stripe) can't take part. sdm w05 day-02-saga-pattern idea W05 · Day 2 · Distributed Transactions — The Saga Pattern
A saga is local steps plus compensations Each step is a local transaction in one service; if step N fails, run compensations for steps 1 to N-1. No distributed locks, no coordinator holding them. sdm w05 day-02-saga-pattern idea W05 · Day 2 · Distributed Transactions — The Saga Pattern
Compensation is not rollback A refund is a new transaction that reverses the charge; both stay in history. Compensations must be idempotent, check state first (no-op if the step never ran) and eventually succeed. sdm w05 day-02-saga-pattern idea W05 · Day 2 · Distributed Transactions — The Saga Pattern
The pivot transaction The point of no return, like shipping. Before it, recover backward (compensate); after it, recover forward (retry until done). In the Airbnb example the pivot is capturing payment. sdm w05 day-02-saga-pattern idea W05 · Day 2 · Distributed Transactions — The Saga Pattern
Choreography vs orchestration Choreography: services react to each other's events; fine for 2–4 steps but the flow is hard to see. Orchestration: a coordinator calls each step and persists saga state; better for 5+ steps and branching. sdm w05 day-02-saga-pattern idea W05 · Day 2 · Distributed Transactions — The Saga Pattern
Numbers to remember

2–4 steps choreography is fine
5+ steps orchestrate
24 hours a host has to accept in the Airbnb booking saga
23 h deadline just inside the 24 h car hold, the first to lapse, in the travel example

sdm w05 day-02-saga-pattern numbers W05 · Day 2 · Distributed Transactions — The Saga Pattern
Backward vs forward recovery

	Backward (compensate)	Forward (retry)
On failure	Undo previous steps	Retry until success
Ends in	The original state	The saga completed
Use	Before the pivot	After the pivot

sdm w05 day-02-saga-pattern tradeoff W05 · Day 2 · Distributed Transactions — The Saga Pattern
Classic mistakes

A refund compensation that can fail with no retry: the saga sticks with the customer charged.
Payment steps without an idempotency key like `saga-{id}-payment`: a timeout retry charges twice.
Compensating everything blindly instead of tracking which compensations finished.

sdm w05 day-02-saga-pattern mistakes W05 · Day 2 · Distributed Transactions — The Saga Pattern
What if the compensation itself fails? Keep trying: record the compensation intent before calling out, use an idempotency key, let a background job retry stuck compensations, and alert humans as the last resort. sdm w05 day-02-saga-pattern ask W05 · Day 3 · Saga Orchestration Deep Dive
Durable execution Temporal records every activity result in an event history. After a crash the workflow replays from the start, completed activities return their stored results instead of running again, and execution continues where it stopped. sdm w05 day-03-saga-orchestration idea W05 · Day 3 · Saga Orchestration Deep Dive
Workflows decide, activities act Workflow code must be deterministic because it replays: no random numbers, clock reads, UUIDs or HTTP calls (use `workflow.now()`, `workflow.random()`, activities). Activities do the side effects, with timeouts and retries. sdm w05 day-03-saga-orchestration idea W05 · Day 3 · Saga Orchestration Deep Dive
Signals and queries A signal delivers an external event (manager approved, rider cancelled) to a waiting workflow, which can sleep for days at no cost. A query reads workflow state without blocking it. sdm w05 day-03-saga-orchestration idea W05 · Day 3 · Saga Orchestration Deep Dive
Version and continue-as-new `workflow.patched("fraud-check-v2")` keeps in-flight workflows on the old path while new ones get the change. Long-lived workflows (subscriptions) use continue-as-new so history stays bounded. sdm w05 day-03-saga-orchestration idea W05 · Day 3 · Saga Orchestration Deep Dive
Numbers to remember

~12 billion Cadence workflow executions a month at Uber (~4,600/s)
~270 billion actions a month (~100,000/s)
$50K loan amount above which the example waits for a manager signal
14 days income verification timeout before auto-reject

sdm w05 day-03-saga-orchestration numbers W05 · Day 3 · Saga Orchestration Deep Dive
Netflix Conductor vs Temporal

	Conductor	Temporal
Workflows defined in	JSON	Code
API	REST	gRPC
Expressiveness	Simpler DSL	Arbitrary code, signals, queries
Fits	Microservice orchestration	Complex business logic

sdm w05 day-03-saga-orchestration tradeoff W05 · Day 3 · Saga Orchestration Deep Dive
Classic mistakes

Calling `datetime.now()` or an HTTP API inside workflow code: replay diverges.
A workflow that loops forever without continue-as-new: unbounded history.
Activities without timeouts, so one hung call stalls the workflow.

sdm w05 day-03-saga-orchestration mistakes W05 · Day 3 · Saga Orchestration Deep Dive
How does Temporal survive a worker crash mid-workflow? Another worker replays the event history: completed activities return recorded results without re-running, and it resumes at the first unfinished step. Activities are at-least-once, so keep them idempotent. sdm w05 day-03-saga-orchestration ask W05 · Day 4 · Conflict Resolution
Last-write-wins loses data The highest timestamp wins, so of two concurrent writes one silently disappears, and clock skew can drop the truly latest one. Fine for a location, a profile picture or "last seen"; wrong for carts and counters. sdm w05 day-04-conflict-resolution idea W05 · Day 4 · Conflict Resolution
Vector clocks detect concurrency Each node keeps a counter per node, increments its own on every write and takes the max on receive. If neither clock is ≤ the other in every entry, the writes were concurrent: a real conflict. sdm w05 day-04-conflict-resolution idea W05 · Day 4 · Conflict Resolution
CRDTs merge by construction Merge is commutative, associative and idempotent, so replicas converge whatever the order. G-Counter sums per-node counts; PN-Counter pairs two; OR-Set tags each add so a concurrent add beats a remove. sdm w05 day-04-conflict-resolution idea W05 · Day 4 · Conflict Resolution
Pick by data semantics Latest is right: LWW. Need to detect and ask: vector clocks with siblings (Riak `allow_mult`). Must merge offline: CRDTs. Collaborative text: OT (Google Docs) or a sequence CRDT (Yjs, Automerge). sdm w05 day-04-conflict-resolution idea W05 · Day 4 · Conflict Resolution
Numbers to remember

100 → 101, not 102 two concurrent increments under read-modify-write
3 + 1 = 4 apples a counter CRDT merging phone and laptop adds
1 entry per server in a vector clock, not one per client
RGA the sequence CRDT behind collaborative text

sdm w05 day-04-conflict-resolution numbers W05 · Day 4 · Conflict Resolution
State-based vs operation-based CRDTs

	State-based (CvRDT)	Operation-based (CmRDT)
Ships	The whole state	Each operation
Bandwidth	More	Less
Needs	Any delivery, merges repeat safely	Reliable, usually causal, delivery

sdm w05 day-04-conflict-resolution tradeoff W05 · Day 4 · Conflict Resolution
Classic mistakes

A counter as an LWW value: concurrent increments vanish. Use a PN-Counter.
A plain set for add and remove: concurrent operations resurrect or drop items. Use an OR-Set.
Never garbage-collecting tombstones, so CRDT state grows forever.

sdm w05 day-04-conflict-resolution mistakes W05 · Day 4 · Conflict Resolution
How do you merge a cart edited on two offline devices? An OR-Set: each add gets a unique tag and a remove tombstones only the tags it saw, so concurrent adds from both devices survive. Add-wins matches what users expect. sdm w05 day-04-conflict-resolution ask W05 · Day 5 · Leader Election and Coordination
Timeouts can't tell slow from dead A GC pause, slow network or clock skew makes a live leader look dead; a follower takes over and two leaders accept writes: split-brain. You can't guarantee both safety and liveness, so favour safety. sdm w05 day-05-leader-election idea W05 · Day 5 · Leader Election and Coordination
Fencing tokens stop zombie leaders Every election issues a higher token. The leader sends it with each request and resources reject anything below the highest they've seen, so a leader waking from a pause can't corrupt data. sdm w05 day-05-leader-election idea W05 · Day 5 · Leader Election and Coordination
Raft in one breath Followers expect heartbeats; on an election timeout one becomes a candidate, bumps the term and asks for votes; a majority makes it leader. At most one leader per term, and terms act as fencing tokens. sdm w05 day-05-leader-election idea W05 · Day 5 · Leader Election and Coordination
Don't build it: ZooKeeper or etcd ZooKeeper's ephemeral sequential znodes make the lowest sequence number the leader, deleted when its session expires; watches announce changes. etcd gives leases on Raft. sdm w05 day-05-leader-election idea W05 · Day 5 · Leader Election and Coordination
Epochs everywhere Kafka elects partition leaders from the in-sync replicas and bumps an epoch so an old leader is rejected; Redis Sentinel needs a quorum to declare the master down; Kubernetes controllers hold a Lease object. sdm w05 day-05-leader-election idea W05 · Day 5 · Leader Election and Coordination
Numbers to remember

10–30 s lease duration; renew at 1/3 of it
10–30 s typical ZooKeeper session timeout, roughly the failover time
3–5 nodes in a ZooKeeper cluster: survives a minority failing
1 MB maximum data in a znode

sdm w05 day-05-leader-election numbers W05 · Day 5 · Leader Election and Coordination
Short vs long leases

	Short lease	Long lease
Failover	Fast	Slow
False elections	More, on network hiccups	Fewer
Risk	Flapping leaders	Long unavailability after a crash

sdm w05 day-05-leader-election tradeoff W05 · Day 5 · Leader Election and Coordination
Classic mistakes

A lock with no fencing token: a paused holder wakes up and writes over the new holder.
"No heartbeat in 5 s, so I'm leader" without asking a quorum.
Leases on wall-clock time; use monotonic clocks and renew early.

sdm w05 day-05-leader-election mistakes W05 · Day 5 · Leader Election and Coordination
How do you prevent split-brain? Elect with a quorum so a minority partition can't choose a leader, give each leader a fencing token resources check, and use leases that expire unless renewed. A zombie's stale token gets rejected. sdm w05 day-05-leader-election ask W05 · Boss · Multiplayer Game Matchmaking System
Size it with Little's law 2M concurrent players, 1.6M in 5v5 matches (160,000 matches of 20 minutes): 8,000 matches start a minute, ~1,300 queue joins a second. At a 30 s wait, only ~40,000 players sit in queue. sdm w05 capstone-interview-week-5-multiplayer-game-matchmaking-system idea W05 · Boss · Multiplayer Game Matchmaking System
One matchmaker per region, fenced Two matchmakers reading the same queue would put the same 10 players in two matches. Elect one leader per region in etcd, and make every queue change carry its fencing token, checked atomically in Redis. sdm w05 capstone-interview-week-5-multiplayer-game-matchmaking-system idea W05 · Boss · Multiplayer Game Matchmaking System
Ratings need strong consistency Write ratings to the primary; a player viewing their own rating gets read-your-writes; the queue snapshots the rating from the primary at join time; leaderboards read cached replicas. sdm w05 capstone-interview-week-5-multiplayer-game-matchmaking-system idea W05 · Boss · Multiplayer Game Matchmaking System
Match creation is a saga Remove players from the queue, ready check, allocate a server, write the match, notify. If allocation fails, compensate by re-queuing confirmed players with priority; a leader crash leaves `pending_match_id` records to clean up. sdm w05 capstone-interview-week-5-multiplayer-game-matchmaking-system idea W05 · Boss · Multiplayer Game Matchmaking System
Party state as an OR-Set Kicks, leaves and invites can happen concurrently on different devices. With an OR-Set the merge is deterministic: concurrent removes of Carol both take effect, Dave's add survives. Parties match on their highest-rated player. sdm w05 capstone-interview-week-5-multiplayer-game-matchmaking-system idea W05 · Boss · Multiplayer Game Matchmaking System
Numbers to remember

±100 ELO preferred range, widened after 60 s
< 30 s · 2 min 5v5 queue time: median, p95
10 s etcd lease TTL before a new leader
10 s grace after a ready-check disconnect, then a 1 min ban

sdm w05 capstone-interview-week-5-multiplayer-game-matchmaking-system numbers W05 · Boss · Multiplayer Game Matchmaking System
Consistency per operation

	Consistency	Read from
Rating update	Linearizable	Primary
My own rating	Read-your-writes	Primary after an update
Matchmaking rating	Linearizable	Primary, at queue join
Leaderboard	Eventual	Cached replica

sdm w05 capstone-interview-week-5-multiplayer-game-matchmaking-system tradeoff W05 · Boss · Multiplayer Game Matchmaking System
Classic mistakes

Two matchmakers without fencing: players double-booked into two games.
Matching on a stale replica's rating right after a match, creating unfair games.
Removing players from the queue with no compensation if server allocation fails.

sdm w05 capstone-interview-week-5-multiplayer-game-matchmaking-system mistakes W05 · Boss · Multiplayer Game Matchmaking System
What breaks first at 10x? The matchmaker cycle: 500 ms per region can't process ~400K queued players, and you can't add leaders within a region. Parallelize by ELO band, shard queues by region and mode, scale stateless services out. sdm w05 capstone-interview-week-5-multiplayer-game-matchmaking-system ask W06 · Day 1 · Designing a Real-Time Notification Platform
Two halves Store-and-forward through providers (push, SMS, email), where "delivered" mostly means a provider accepted it; and a real-time tier where we hold a connection to online users and deliver ourselves in under a second. sdm w06 day-01-notification-platform-design idea W06 · Day 1 · Designing a Real-Time Notification Platform
Five classes, different clocks Critical (OTP, fraud), transactional (payments), social, reminder and marketing each have their own latency, channels and quiet-hours rule, and the lane follows. Marketing must never delay a transaction. sdm w06 day-01-notification-platform-design idea W06 · Day 1 · Designing a Real-Time Notification Platform
Inbox first, then the hint In-app delivery writes the item to a durable per-user inbox with a sequence number, then sends the gateway a hint. Lose the hint and the client resumes "after 1183". No ack within seconds: push. sdm w06 day-01-notification-platform-design idea W06 · Day 1 · Designing a Real-Time Notification Platform
OTPs skip the ladder A code is useless late, so an OTP goes by SMS at once. Security alerts start with push and escalate to SMS if still unread after 2 minutes. Critical ignores opt-outs and quiet hours. sdm w06 day-01-notification-platform-design idea W06 · Day 1 · Designing a Real-Time Notification Platform
Sent is not delivered FCM counts a send once the message is "enqueued for delivery". Keep one deliveries row per channel attempt, separate from the notification and the inbox item, and keep `sent` and `delivered` apart. sdm w06 day-01-notification-platform-design idea W06 · Day 1 · Designing a Real-Time Notification Platform
Numbers to remember

~5,800/s average for 500M a day; design for 20,000
~89% of the provider bill is SMS, 5% of messages
~10M live connections: 15% of 50M online, ~1.33 devices each
70 characters per UCS-2 segment; "₹" isn't in GSM-7

sdm w06 day-01-notification-platform-design numbers W06 · Day 1 · Designing a Real-Time Notification Platform
Four channels

	Push	SMS	Email	In-app
Cost	Free	Per segment	$0.10 per 1,000	Our compute
Reach	Needs permission	Any phone	Any address	App open now
Success means	Accepted	Carrier receipt	Accepted, webhooks later	Client ack

sdm w06 day-01-notification-platform-design tradeoff W06 · Day 1 · Designing a Real-Time Notification Platform
Classic mistakes

One queue for every class: the OTP that leaves at 20:08.
Treating a provider's 200 as delivery.
The socket as the record: every crash, dropped Wi-Fi and deploy loses messages.

sdm w06 day-01-notification-platform-design mistakes W06 · Day 1 · Designing a Real-Time Notification Platform
Why an inbox if you have push? Push is a doorbell with little offline storage and no ordering. The inbox is the record that clients read and resume from. sdm w06 day-01-notification-platform-design ask W06 · Day 2 · The Delivery Pipeline: Guarantees, Priorities and Preferences
Key the business event The idempotency key names the event (`payments:pay_8812:received`), not the HTTP request, and commits with the notification and its outbox row. A retry gets the original id; a reused key with different parameters gets `422`. sdm w06 day-02-core-notification-flow idea W06 · Day 2 · The Delivery Pipeline: Guarantees, Priorities and Preferences
Check late, then again The router reads cached preferences, but the channel worker re-checks opt-out, quiet hours, `expires_at` and business conditions just before the provider call, marking failures suppressed or expired. sdm w06 day-02-core-notification-flow idea W06 · Day 2 · The Delivery Pipeline: Guarantees, Priorities and Preferences
Lanes all the way down Critical, transactional and bulk each get their own topic, consumer group, workers, retry topics and share of provider quota. One topic with a priority field can't skip a partition's backlog. sdm w06 day-02-core-notification-flow idea W06 · Day 2 · The Delivery Pipeline: Guarantees, Priorities and Preferences
At-least-once, deduplicated Exactly-once isn't available through a provider you don't control. Claim, send, record per task; dedup at every hop; collapse ids hide the rare duplicate. Bulk treats an unconfirmed claim as sent. sdm w06 day-02-core-notification-flow idea W06 · Day 2 · The Delivery Pipeline: Guarantees, Priorities and Preferences
Escalation is a forward-only saga Fraud alert: push now, SMS if unread at 2 minutes, email 10 minutes later, fraud team at 24 hours. Durable timers, not `sleep(120)`; "undo" cancels the steps not yet taken. sdm w06 day-02-core-notification-flow idea W06 · Day 2 · The Delivery Pipeline: Guarantees, Priorities and Preferences
Numbers to remember

5 min · 120 s OTP code validity · its SMS validity period
15 → 23 OTP SMS workers by Little's law; with a zone lost
~22 min 8M campaign pushes at 6,000/s
60 s FCM's default wait after a `429`

sdm w06 day-02-core-notification-flow numbers W06 · Day 2 · The Delivery Pipeline: Guarantees, Priorities and Preferences
Delivery semantics per lane

	At-most-once	At-least-once + dedup
How	Mark done, then send	Send, then mark done
Crash in between	Message lost	Rare duplicate
Fits	Bulk marketing	Critical, transactional

sdm w06 day-02-core-notification-flow tradeoff W06 · Day 2 · The Delivery Pipeline: Guarantees, Priorities and Preferences
Classic mistakes

Promising exactly-once instead of at-least-once with dedup.
Checking preferences only at routing, so opt-outs lag behind the queue.
Retrying a `410`, or delivering an expired OTP.

sdm w06 day-02-core-notification-flow mistakes W06 · Day 2 · The Delivery Pipeline: Guarantees, Priorities and Preferences
An OTP arrives mid-campaign. What protects it? A different topic, its own workers and reserved provider quota, so it never sees the campaign. The campaign is quota-bound, so it's throttled and its lag is expected. sdm w06 day-02-core-notification-flow ask W06 · Day 3 · Real-Time Channels: WebSockets and SSE at Scale
Pick the transport per surface SSE for the one-way web bell: free reconnect and `Last-Event-ID` resume, and HTTP/2 lifts the 6-connection cap. WebSockets for apps that ack. Long polling as the last resort. sdm w06 day-03-advanced-features idea W06 · Day 3 · Real-Time Channels: WebSockets and SSE at Scale
Size by connections 10M sockets heartbeating every 30 s is ~333,000 beats a second against ~1,700 notifications at peak. Cap nodes at 200,000 for blast radius: 50 nodes, 75 to survive a zone. sdm w06 day-03-advanced-features idea W06 · Day 3 · Real-Time Channels: WebSockets and SSE at Scale
Registry with leases and epochs The registry maps user and device to gateway and epoch. Each gateway holds a 15-second lease; lookups skip dead leases and old epochs. One pub/sub channel per gateway. sdm w06 day-03-advanced-features idea W06 · Day 3 · Real-Time Channels: WebSockets and SSE at Scale
The socket is a hint Redis Pub/Sub is at-most-once, which is fine because the inbox already has the item. Clients resume "after seq N" with a range read; no ack within seconds falls back to push. sdm w06 day-03-advanced-features idea W06 · Day 3 · Real-Time Channels: WebSockets and SSE at Scale
Heartbeat under every idle timeout TCP keepalive waits 2 hours; one carrier NAT cut idle TCP at 255 s; an ALB's default is 60 s. Client heartbeat every 25–30 s with jitter; server closes after ~2.5 misses. sdm w06 day-03-advanced-features idea W06 · Day 3 · Real-Time Channels: WebSockets and SSE at Scale
Shape the reconnect storm Full-jitter backoff with a jittered first attempt, connect admission control answering "retry after", resume instead of reboot, bounded per-socket queues, and slow drains: 1% every 6 seconds. sdm w06 day-03-advanced-features idea W06 · Day 3 · Real-Time Channels: WebSockets and SSE at Scale
Numbers to remember

200,000 connections per gateway node, a blast-radius cap
~3.3M sockets reconnect after a zone loss: ~67 s if spread evenly
75 s server idle timeout: ~2.5 missed 30-second beats
17% of Discord's sessions dropped in its March 2026 deploy

sdm w06 day-03-advanced-features numbers W06 · Day 3 · Real-Time Channels: WebSockets and SSE at Scale
Routing to a gateway

	Sticky hash	Connection registry
Hot-path lookup	None	One, sub-millisecond
Dead node's clients	Pile onto one successor	Spread across the fleet
Multi-device	Awkward	Each device an entry

sdm w06 day-03-advanced-features tradeoff W06 · Day 3 · Real-Time Channels: WebSockets and SSE at Scale
Classic mistakes

Using the socket as the record.
Unbounded per-connection buffers: a few slow phones run the node out of memory.
Fixed-delay reconnects that recreate the wave every 2 seconds.

sdm w06 day-03-advanced-features mistakes W06 · Day 3 · Real-Time Channels: WebSockets and SSE at Scale
A gateway dies. What do its users see? A dropped socket, a jittered reconnect elsewhere and a resume from the inbox. The registry ignores the gateway once its lease expires, and no ack in time means push. sdm w06 day-03-advanced-features ask W06 · Day 4 · Presence and Chat Fan-Out at Scale
Three classes of event Durable messages are stored first, ordered and retried. Presence is soft state in memory with a TTL, latest wins. Typing is ephemeral: throttled, expired by the receiver, dropped first. sdm w06 day-04-scale-reliability-edge-cases idea W06 · Day 4 · Presence and Chat Fan-Out at Scale
Presence expires by itself Heartbeat every d seconds; the entry expires after d + ε, and a delayed trigger at d + 2ε publishes offline only if no newer beat came. Last seen uses the server's clock. sdm w06 day-04-scale-reliability-edge-cases idea W06 · Day 4 · Presence and Chat Fan-Out at Scale
Tell only who's looking Clients subscribe to the users on their screen. Eager presence costs ~84,000 pushes a second, twice the message traffic; visible-only with debouncing, ~22,000, about half. sdm w06 day-04-scale-reliability-edge-cases idea W06 · Day 4 · Presence and Chat Fan-Out at Scale
Bodies once, pointers to small groups The body goes once into the conversation log; small groups get a pointer in each member's inbox; huge rooms get no per-member writes, and full live pushes only reach open sessions. sdm w06 day-04-scale-reliability-edge-cases idea W06 · Day 4 · Presence and Chat Fan-Out at Scale
One owner sequences each conversation The owner assigns 1, 2, 3…, never the sender's clock. A new owner starts from the durable max, and a unique (conversation_id, seq) key fails a stale owner's write. sdm w06 day-04-scale-reliability-edge-cases idea W06 · Day 4 · Presence and Chat Fan-Out at Scale
Receipts are cursors "Delivered up to 412" and "read up to 410", applied as `max(old, new)`, so ticks never move backwards. A group's tick is the minimum cursor over its members. sdm w06 day-04-scale-reliability-edge-cases idea W06 · Day 4 · Presence and Chat Fan-Out at Scale
Numbers to remember

~84,000 → ~22,000 presence pushes a second, eager vs visible-only
3 s · 5 s typing throttle · receiver expiry
1,024 WhatsApp's group size cap
30 s Discord's delayed Read State commit

sdm w06 day-04-scale-reliability-edge-cases numbers W06 · Day 4 · Presence and Chat Fan-Out at Scale
Where to do fan-out work

	Write fan-out	Read fan-out
Send	N inbox writes	One append
Sync	One inbox, one cursor	One query per conversation
Breaks at	300,000-member groups	Many quiet conversations

sdm w06 day-04-scale-reliability-edge-cases tradeoff W06 · Day 4 · Presence and Chat Fan-Out at Scale
Classic mistakes

Writing presence to the message database like messages.
Broadcasting presence to every contact.
Ordering messages by the sender's timestamp.

sdm w06 day-04-scale-reliability-edge-cases mistakes W06 · Day 4 · Presence and Chat Fan-Out at Scale
Can you guarantee exactly-once delivery? At-least-once with dedup on both ends: sends on (device, client message id), receives on (conversation, seq). That gives exactly-once display. sdm w06 day-04-scale-reliability-edge-cases ask W06 · Day 5 · Operations, Scale and Edge Cases
Measure the user's clock End-to-end latency from `created_at`, not dequeue time, and the oldest message's age per lane. Success rates only count attempts, so a four-minute queue looks perfect. sdm w06 day-05-operations-monitoring-interview idea W06 · Day 5 · Operations, Scale and Edge Cases
SLOs stop at the provider We can't see an iPhone show a banner. Critical: 99.9% handed off, or acked in-app, within 10 seconds. Bulk has no availability target: pausing a campaign is the system working. sdm w06 day-05-operations-monitoring-interview idea W06 · Day 5 · Operations, Scale and Edge Cases
Lanes reach the provider FCM's quota is per project and an SMS sender has one queue. Reserve quota that bulk can't touch, use separate SMS senders for OTPs and marketing, and short validity periods. sdm w06 day-05-operations-monitoring-interview idea W06 · Day 5 · Operations, Scale and Edge Cases
Shift SMS in steps GOV.UK Notify's failover moved a spike onto a cold provider, which failed too. Always split about 50/50; a 5xx cuts that provider's share 10 points, at most once a minute. sdm w06 day-05-operations-monitoring-interview idea W06 · Day 5 · Operations, Scale and Edge Cases
Same code on every attempt A timeout may have delivered, so failover can send twice. Create the OTP once and key the ledger `(notification_id, channel)`, not by provider, so a duplicate text is harmless. sdm w06 day-05-operations-monitoring-interview idea W06 · Day 5 · Operations, Scale and Edge Cases
Numbers to remember

~14 min stall at peak that spends the critical lane's monthly budget
600,000/min FCM's default quota per project
160 · 70 characters per segment, GSM-7 vs UCS-2
~$11.9M → ~$5.9M monthly SMS at list price, before and after two levers

sdm w06 day-05-operations-monitoring-interview numbers W06 · Day 5 · Operations, Scale and Edge Cases
When a provider fails

	SMS	APNs or FCM
Second provider	Yes, always split	None
During an outage	Shift share in steps	Hold, in-app, SMS for critical
On recovery	Drift back hourly	Ramp the drain, critical first

sdm w06 day-05-operations-monitoring-interview tradeoff W06 · Day 5 · Operations, Scale and Edge Cases
Classic mistakes

Measuring the worker, not the user.
Counting bad tokens as provider failures, opening a breaker on a healthy APNs.
No owner for SMS spend: a retry loop or pumping shows up on next month's invoice.

sdm w06 day-05-operations-monitoring-interview mistakes W06 · Day 5 · Operations, Scale and Edge Cases
APNs is down. What do you do? Hold with TTLs, use the socket for users with the app open, SMS for critical only, let bulk wait or expire, and ramp the drain on recovery. sdm w06 day-05-operations-monitoring-interview ask W06 · Boss · Food Delivery Order System
Size the dinner rush 5M orders a day is ~58 a second; dinner is 4×, ~230. That's ~7,000 writes and ~17,000 browsing reads a second, and ~400K active orders to track. sdm w06 capstone-interview-week-6-food-delivery-system idea W06 · Boss · Food Delivery Order System
Never lose an order Write the order and an outbox row in one transaction; the publisher marks rows published only after Kafka confirms, so a crash republishes and idempotent consumers absorb the duplicate. sdm w06 capstone-interview-week-6-food-delivery-system idea W06 · Boss · Food Delivery Order System
Payments: authorize, capture, refund Authorize at order time, capture when the driver picks up, refund on cancellation, every Stripe call with an idempotency key, so a retried authorization can't create a second PaymentIntent. sdm w06 capstone-interview-week-6-food-delivery-system idea W06 · Boss · Food Delivery Order System
No driver assigned twice Two orders can both see Driver X as free. Assign under a row lock with a version check, so one succeeds and the other tries its next candidate. sdm w06 capstone-interview-week-6-food-delivery-system idea W06 · Boss · Food Delivery Order System
Push live tracking, don't poll Polling 400K orders every 10 s is ~40,000 mostly empty reads a second. One SSE or WebSocket connection per screen: location coalesced to the latest point, status resumed by sequence number. sdm w06 capstone-interview-week-6-food-delivery-system idea W06 · Boss · Food Delivery Order System
Order updates through the lanes Order updates are transactional: inbox first, a live hint, push only if no device acks. OTPs and driver offers are critical; promotions are bulk. Driver online status is presence, never used for correctness. sdm w06 capstone-interview-week-6-food-delivery-system idea W06 · Boss · Food Delivery Order System
Numbers to remember

< 2 s restaurant sees the order
< 30 s driver assignment after restaurant confirmation
~600K live connections at the dinner peak: ~5 gateway nodes
3–5 s driver location update interval

sdm w06 capstone-interview-week-6-food-delivery-system numbers W06 · Boss · Food Delivery Order System
Tracking: poll or push

	Poll every 10 s	Live connection
Load	~40,000 reads/s	~600K connections
Status latency	Up to 10 s	About a second
After a drop	Next poll	Resume after last seq

sdm w06 capstone-interview-week-6-food-delivery-system tradeoff W06 · Boss · Food Delivery Order System
Classic mistakes

Saving the order, then publishing to Kafka: Kafka down means a held card and no food.
Retrying payment calls without an idempotency key.
Reassigning an order because a driver's presence dot went grey.

sdm w06 capstone-interview-week-6-food-delivery-system mistakes W06 · Boss · Food Delivery Order System
How does this scale to 10x (50M orders a day)? Shard orders, drivers and restaurants by city; add read replicas and cache; Redis Cluster for locations; more Kafka partitions; ~300 API servers; ~45 gateway nodes for ~6M connections. sdm w06 capstone-interview-week-6-food-delivery-system ask W07 · Day 1 · Search Fundamentals & Architecture
Why LIKE fails `LIKE '%red%'` can't use a B-tree, which is sorted by string start, so 50M rows get a full scan. It also has no relevance ranking, typo tolerance, word boundaries, stemming or synonyms. sdm w07 day-01-search-fundamentals idea W07 · Day 1 · Search Fundamentals & Architecture
The inverted index Pre-compute word to documents instead of scanning documents for words. A query is one lookup per term plus an intersection, smallest list first: "running shoes" takes under a millisecond. sdm w07 day-01-search-fundamentals idea W07 · Day 1 · Search Fundamentals & Architecture
Text analysis Character filters (strip HTML), then the tokenizer, then token filters (lowercase, stop words, stemming, synonyms). The same analyzer must run at index time and query time, or "Running" never matches the indexed "run". sdm w07 day-01-search-fundamentals idea W07 · Day 1 · Search Fundamentals & Architecture
The multi-field pattern One source field, several representations: `brand` as `text` for search, `brand.keyword` for facets and sorting, `brand.autocomplete` for typeahead. IDs are `keyword`: they're only exact-matched. sdm w07 day-01-search-fundamentals idea W07 · Day 1 · Search Fundamentals & Architecture
Scatter-gather A coordinating node sends the query to every shard (primary or replica). Each searches its own inverted index and returns its top N with scores; the coordinator merges, sorts by score and returns the overall top N. sdm w07 day-01-search-fundamentals idea W07 · Day 1 · Search Fundamentals & Architecture
The product search cluster 20 primary shards plus 20 replicas on 6 data nodes, 3 dedicated master nodes, 2 coordinating nodes. Text goes in `must` to drive the score; category, price and stock go in `filter`. sdm w07 day-01-search-fundamentals idea W07 · Day 1 · Search Fundamentals & Architecture
Numbers to remember

30+ seconds a `LIKE` scan over 50M products
50M × 10 KB about 500 GB of raw data
20–40 GB per shard; max ~50 GB, min 1 GB
64 GB RAM per data node: ~26 GB heap, the rest OS cache

sdm w07 day-01-search-fundamentals numbers W07 · Day 1 · Search Fundamentals & Architecture
keyword vs text

	`keyword`	`text`
Analysis	None, stored as-is	Tokenized, normalized
Supports	Exact match, aggregations, sorting	Full-text queries, relevance
Use for	IDs, categories, tags	Names, descriptions

sdm w07 day-01-search-fundamentals tradeoff W07 · Day 1 · Search Fundamentals & Architecture
Classic mistakes

Different analyzers at index and query time, so tokens never match.
Oversharding: per-shard memory and file-handle overhead degrades queries.
Not planning primary shards: the count is fixed at creation, so changing it needs `_split`/`_shrink` or a reindex.

sdm w07 day-01-search-fundamentals mistakes W07 · Day 1 · Search Fundamentals & Architecture
How would you size shards for a 500 GB index? 500 GB at 30 GB per shard is about 17, rounded to 20 primary shards; one replica makes 40 copies. Huge shards recover slowly; hundreds of tiny ones are oversharding. sdm w07 day-01-search-fundamentals ask W07 · Day 2 · Indexing Pipeline
Never dual-write Writing PostgreSQL then Elasticsearch breaks four ways: the ES write fails, the process crashes between writes, a partition holds the transaction open, or retries reorder updates so ES ends at $100 while PostgreSQL has $120. sdm w07 day-02-indexing-pipeline idea W07 · Day 2 · Indexing Pipeline
Change data capture App to PostgreSQL; Debezium reads the WAL through logical replication and publishes change events to Kafka; the indexing pipeline consumes them into Elasticsearch. The app writes to one place, and events replay within Kafka's retention. sdm w07 day-02-indexing-pipeline idea W07 · Day 2 · Indexing Pipeline
Version by WAL position Each bulk operation carries `version: event.lsn` with `version_type: external`. If an older change arrives late after retries or reordered batches, ES rejects it with a version conflict instead of overwriting newer data. sdm w07 day-02-indexing-pipeline idea W07 · Day 2 · Indexing Pipeline
Batch, but by priority Flush at 500 documents or after 5 seconds, so quiet periods don't strand updates. Stock going to zero is CRITICAL in 10-document batches; description edits are NORMAL at 500; bulk imports LOW at 1,000. sdm w07 day-02-indexing-pipeline idea W07 · Day 2 · Indexing Pipeline
Zero-downtime reindex Reads stay on the `products` alias over v1 while updates go to both indexes and old data is copied with `op_type: create`. Verify counts, switch the alias in one atomic remove+add, then delete v1. sdm w07 day-02-indexing-pipeline idea W07 · Day 2 · Indexing Pipeline
Numbers to remember

1M at 100/s about 3 hours through the normal CDC path
5,000/s bulk indexer rate limit, in 1,000-document batches
< 60 s · < 1% healthy indexing lag and error rate
0.1% max count gap before the alias switch

sdm w07 day-02-indexing-pipeline numbers W07 · Day 2 · Indexing Pipeline
Dual-write vs CDC

	Dual-write	CDC
App knows about ES	Yes	No
One write fails	Stores diverge	Event waits in Kafka
Ordering	Retries can reorder	Kafka order per partition
Replay	None	From Kafka

sdm w07 day-02-indexing-pipeline tradeoff W07 · Day 2 · Indexing Pipeline
Classic mistakes

Running a big import through the live pipeline, starving CDC updates.
Skipping index aliases, so a reindex can't switch with zero downtime.
Blaming the pipeline when the ES document is right but a result cache is stale.

sdm w07 day-02-indexing-pipeline mistakes W07 · Day 2 · Indexing Pipeline
How do you keep search in sync with the database? CDC, not dual-write: read the transaction log, publish to Kafka, and let a separate pipeline update Elasticsearch. A rate-limited bulk path handles imports, and aliases allow zero-downtime reindexing. sdm w07 day-02-indexing-pipeline ask W07 · Day 3 · Query Processing & Relevance
BM25 in one line Score grows with term frequency but saturates (`k1`, default 1.2), rare terms weigh more (IDF), and long documents are normalized (`b`, default 0.75). 10 occurrences aren't 10x better than 1. sdm w07 day-03-query-processing-relevance idea W07 · Day 3 · Query Processing & Relevance
Rare terms win In the example corpus, "nike" (2% of documents) has IDF 3.912, "shoes" (10%) 2.3026, and "the" (90%) 0.1054. A match on the rarer term says more about relevance. sdm w07 day-03-query-processing-relevance idea W07 · Day 3 · Query Processing & Relevance
Query context vs filter context What the user typed is scored in query context. What they selected (facets, toggles, ranges) goes in filter: yes or no, unscored, cacheable as a bitset. The lesson puts the split at ~3x faster. sdm w07 day-03-query-processing-relevance idea W07 · Day 3 · Query Processing & Relevance
Boost in layers Start with field boosts (`name^3`, `brand^2`, `description`), add business boosts in `should` clauses, and use `function_score` (popularity, recency decay, rating, featured) only when needed. A/B test every ranking change. sdm w07 day-03-query-processing-relevance idea W07 · Day 3 · Query Processing & Relevance
Never return nothing Progressive relaxation: exact AND query, then drop low-priority filters (rating, price), then fuzzy matching, then any-term matching, then no filters at all. Category stays: it's a strong intent signal. sdm w07 day-03-query-processing-relevance idea W07 · Day 3 · Query Processing & Relevance
Pick the multi_match type `best_fields` with `tie_breaker` 0.3 is the product default. `cross_fields` lets "John" match `first_name` and "Smith" `last_name`. `most_fields` sums scores across analyses. `phrase_prefix` and `bool_prefix` serve autocomplete. sdm w07 day-03-query-processing-relevance idea W07 · Day 3 · Query Processing & Relevance
Numbers to remember

1.2 · 0.75 BM25 defaults for `k1` and `b`
50 ms to 1 ms a price filter before and after its bitset is cached
0.2 `negative_boost`: out-of-stock scores drop to 20%
< 5% · > 0.7 excellent zero-result rate and MRR

sdm w07 day-03-query-processing-relevance numbers W07 · Day 3 · Query Processing & Relevance
Query vs filter context

	Query	Filter
Asks	How well does it match?	Does it match?
Score	Calculated	None
Caching	Not cached	Bitset, once frequent
Use for	Search text	Facets, ranges, booleans

sdm w07 day-03-query-processing-relevance tradeoff W07 · Day 3 · Query Processing & Relevance
Classic mistakes

Putting category, brand and price in `must`: they sway scores without adding relevance and aren't cached.
Filtering out-of-stock items away when they should be demoted with a `boosting` query.
Over-boosting, which makes relevance unpredictable.

sdm w07 day-03-query-processing-relevance mistakes W07 · Day 3 · Query Processing & Relevance
How would you improve search relevance? Measure first: zero-result rate, CTR, MRR. Then field boosts, query/filter separation and function scores for popularity, recency and ratings; fuzzy fallback, filter relaxation and spelling suggestions. A/B test every change. sdm w07 day-03-query-processing-relevance ask W07 · Day 4 · Typeahead at Scale: Autocomplete, Spelling and Facets
Typeahead multiplies traffic One request per keystroke: a search is ~12 requests. The store's ~2,800 searches a second at peak become ~33,000 typeahead requests, so typeahead is its own cheap system, not prefix queries on the search cluster. sdm w07 day-04-advanced-search-features idea W07 · Day 4 · Typeahead at Scale: Autocomplete, Spelling and Facets
Store the answer, not the means A plain trie ranks whole subtrees, largest for short prefixes. Precompute each prefix's top-k at build time: serving is one walk or map lookup, O(prefix length). An FST is the compressed, immutable version. sdm w07 day-04-advanced-search-features idea W07 · Day 4 · Typeahead at Scale: Autocomplete, Spelling and Facets
Suggestions come from query logs Normalize, count searches and distinct users, decay with a half-life (14 days, illustrative), filter, then ship a versioned snapshot nightly. A stream job adds a small trending overlay, capped in slots and filtered more strictly. sdm w07 day-04-advanced-search-features idea W07 · Day 4 · Typeahead at Scale: Autocomplete, Spelling and Facets
Suggestions are the system speaking A distinct-user minimum keeps one person's phone number out. PII patterns, blocklists and policy lists drop the rest, including unconfirmed rumours. A serve-time blocklist plus CDN purge removes misses in seconds. sdm w07 day-04-advanced-search-features idea W07 · Day 4 · Typeahead at Scale: Autocomplete, Spelling and Facets
Global at the edge, history private The global list is the same for everyone per prefix and locale, so the CDN caches it on that key. History comes privately, merged on the client. Never cache a blended list. sdm w07 day-04-advanced-search-features idea W07 · Day 4 · Typeahead at Scale: Autocomplete, Spelling and Facets
After Enter Spelling: pick the c maximizing P(c) x P(w | c), with your query logs as the language model. Facets: selections in `post_filter`, each facet filtered by every selection except its own, so Adidas stays countable. sdm w07 day-04-advanced-search-features idea W07 · Day 4 · Typeahead at Scale: Autocomplete, Spelling and Facets
Numbers to remember

~100 ms · ~15 ms keystroke to screen; the server's p99 share
12x keystroke requests per search, before debouncing
~4,400/s reach the origin after debounce and a 60% CDN hit ratio
~11 GB the whole suggestion index, so replicate, don't shard

sdm w07 day-04-advanced-search-features numbers W07 · Day 4 · Typeahead at Scale: Autocomplete, Spelling and Facets
Where suggestions come from

	Precomputed top-k	Edge n-grams
Lookup	One key, microseconds	A scored, sharded query
Filters	Only those built in	Any, live
Best for	Query suggestions at high QPS	Low-QPS boxes needing filters

sdm w07 day-04-advanced-search-features tradeoff W07 · Day 4 · Typeahead at Scale: Autocomplete, Spelling and Facets
Classic mistakes

A prefix query on the search cluster per keystroke: a sale night slows search for everyone.
Raw all-time counts: old models win forever and one user's query leaks.
Sharding by first letter: skew, and launches make one shard hot.

sdm w07 day-04-advanced-search-features mistakes W07 · Day 4 · Typeahead at Scale: Autocomplete, Spelling and Facets
How would you implement autocomplete? A precomputed prefix-to-top-k service built nightly from logs: distinct users, decay, PII and policy filters, plus a trending overlay. Debounce on the client, cache globally at the CDN, keep history private, replicate the in-memory index. sdm w07 day-04-advanced-search-features ask W07 · Day 5 · Location Search and Search at Scale
Why a B-tree can't do "near" An index on `(lat, lng)` uses only the latitude range, then tests longitude on every row: a strip across the country. Instead, cover the query with cells, then refine by exact distance. sdm w07 day-05-search-operations-scale idea W07 · Day 5 · Location Search and Search at Scale
Geohash and its edge Each base-32 character adds 5 bits and divides the cell by 32. A shared prefix means close, not the reverse: points 43 m apart can share only `tdr`. Search the cell plus 8 neighbours, then refine. sdm w07 day-05-search-operations-scale idea W07 · Day 5 · Location Search and Search at Scale
S2 and H3 S2 numbers a quadtree on a cube in Hilbert order: a cell's descendants are one ID range, ideal for coverings and shard keys. H3 hexagons have six equidistant neighbours, so rings approximate circles; children don't nest exactly. sdm w07 day-05-search-operations-scale idea W07 · Day 5 · Location Search and Search at Scale
Shapes in a database PostGIS puts an R-tree on GiST that stores bounding boxes; `ST_DWithin` uses it, `ST_Distance` in WHERE doesn't. Delivery zones are polygons: ask which zones contain the customer's point. sdm w07 day-05-search-operations-scale idea W07 · Day 5 · Location Search and Search at Scale
Moving objects are soft state Overwrite each partner's position in memory on every ping; touch the cell index only on a cell change. A TTL drops dead phones. The trail goes to Kafka, never into the search index. sdm w07 day-05-search-operations-scale idea W07 · Day 5 · Location Search and Search at Scale
Stop kNN with a bound Grow H3 rings outward, but stop only when you have k results and the k-th is no farther than a lower bound on anything unscanned. Re-rank the top few by road ETA. sdm w07 day-05-search-operations-scale idea W07 · Day 5 · Location Search and Search at Scale
Numbers to remember

75,000/s position writes: 300,000 partners every 4 s
1 in 45 pings that change an H3 res-8 cell
~26 GB safe heap, at most half of RAM (up to 30 GB)
10–50 GB per shard; 32 primaries here

sdm w07 day-05-search-operations-scale numbers W07 · Day 5 · Location Search and Search at Scale
Stores vs partners

	POI index	Live index
Writes	A few a second	Every partner, every 4 s
Freshness	Minutes is fine	Seconds
Home	Elasticsearch, PostGIS	Memory: cells or Redis GEO

sdm w07 day-05-search-operations-scale tradeoff W07 · Day 5 · Location Search and Search at Scale
Classic mistakes

Updating a search document on every ping: reindexing and segment churn slow product search.
Trusting the prefix: missing a partner 43 m away across a cell edge.
One shard per city: one stadium slows the whole city.

sdm w07 day-05-search-operations-scale mistakes W07 · Day 5 · Location Search and Search at Scale
How would you find the nearest drivers? Positions in memory, bucketed by H3 cell, with a TTL. Grow rings until the k-th result beats the unscanned bound, then re-rank by ETA. Shard by load-sized cell ranges; cache hot cells briefly. sdm w07 day-05-search-operations-scale ask W07 · Boss · Designing a Legal Document Search System
Size the corpus 100M documents (50M public, 50M private) at ~65 KB is 6.5 TB of text. 2x index overhead and 2x for replicas make 26 TB in Elasticsearch, plus 50 TB of original PDFs in S3: ~76 TB. sdm w07 capstone-interview-week-7-legal-document-search idea W07 · Boss · Designing a Legal Document Search System
Size the traffic 50,000 concurrent users at 20 searches an hour is 1M an hour, ~280 a second; autocomplete at 5 keystrokes per search is ~1,400 a second. Today's 10,000 a minute (~170/s) leaves ~1.7x headroom. sdm w07 capstone-interview-week-7-legal-document-search idea W07 · Boss · Designing a Legal Document Search System
A real query parser A lexer and parser turn `(breach /5 contract) AND damages NOT punitive` into `must`, `should` and `must_not` clauses. Proximity becomes `span_near` with `in_order: false`; `/s` means ~15 words, `/p` ~50. sdm w07 capstone-interview-week-7-legal-document-search idea W07 · Boss · Designing a Legal Document Search System
Isolation in layers Per-tenant indices, a service-injected `tenant_id` filter, document-level security, a tenant taken from the validated JWT (never the request body), and audit logs. Zero tolerance means no single control is trusted alone. sdm w07 capstone-interview-week-7-legal-document-search idea W07 · Boss · Designing a Legal Document Search System
Citations as edges A separate index of `{citing_id, cited_id, treatment}` makes "who cites X" a term query on `cited_id`, with aggregations for treatment breakdowns, "good law" checks and 2-hop chains. sdm w07 capstone-interview-week-7-legal-document-search idea W07 · Boss · Designing a Legal Document Search System
Triage every upload Store the original in S3 first. A PDF with under 100 characters of text in its first pages is scanned: render each page at 300 DPI and OCR it. Word goes through Tika. Then publish to Kafka. sdm w07 capstone-interview-week-7-legal-document-search idea W07 · Boss · Designing a Legal Document Search System
Numbers to remember

< 500 ms · < 2 s p99 for simple and complex boolean search
< 5 minutes upload to searchable; court opinions < 1 hour
26 TB Elasticsearch footprint with replicas
12 data nodes with 64 GB RAM and 4 TB SSD each

sdm w07 capstone-interview-week-7-legal-document-search numbers W07 · Boss · Designing a Legal Document Search System
Today vs 10x

	Today	10x
Documents	100M	1B
Data nodes	12	~90, or larger instances
Storage	26 TB	260 TB
Peak QPS	300	3,000

Split indices by type and date so queries hit only relevant ones.

sdm w07 capstone-interview-week-7-legal-document-search tradeoff W07 · Boss · Designing a Legal Document Search System
Classic mistakes

Trusting one isolation control, or a `tenant_id` from the request body.
Using a phrase query for proximity: "liability for negligence" should match `negligence /10 liability`.
Fuzzy matching citation variants instead of a canonical case ID plus name synonyms.

sdm w07 capstone-interview-week-7-legal-document-search mistakes W07 · Boss · Designing a Legal Document Search System
What would you build first? Tenant isolation, because security is non-negotiable; then boolean search, document ingestion with OCR, and citation extraction and linking. Proximity search and analytics come once the core is solid. sdm w07 capstone-interview-week-7-legal-document-search ask W08 · Day 1 · Event Ingestion at Scale
Events are immutable facts Each event has a unique `event_id`, an event time, a type and context. Never update one: record the change as a new event (`event.processed`) pointing to the original, or replay and the audit trail break. sdm w08 day-01-event-ingestion idea W08 · Day 1 · Event Ingestion at Scale
Evolve schemas safely A registry enforcing BACKWARD compatibility rejects a new required field like `session_id`, since old events lack it. Add fields as optional with a default. Name events `domain.entity.action`, in the past tense. sdm w08 day-01-event-ingestion idea W08 · Day 1 · Event Ingestion at Scale
Collect, validate, then Kafka Lightweight collectors validate size, required fields, schema, timestamps, business rules and security, then write to Kafka asynchronously. In a batch, valid events go to Kafka and invalid ones to the dead letter topic. sdm w08 day-01-event-ingestion idea W08 · Day 1 · Event Ingestion at Scale
Timestamps and enrichment An event time over 5 minutes in the future is an error; over 30 days old is only a warning, since offline phones send those. Enrichment (geo, user, product, session) is best-effort: a failed lookup never blocks. sdm w08 day-01-event-ingestion idea W08 · Day 1 · Event Ingestion at Scale
Key by user, write idempotently Kafka orders only within a partition, so user events are keyed by `user_id`. `acks=all` makes a write durable across in-sync replicas; `enable_idempotence` stops a retried send from writing a duplicate. sdm w08 day-01-event-ingestion idea W08 · Day 1 · Event Ingestion at Scale
Shed telemetry, never revenue Above 90% queue usage the collector sheds `page.viewed`-type telemetry but never orders, payments or sign-ups. Per-client limits (1,000 a second) and an adaptive limiter protect downstream systems. sdm w08 day-01-event-ingestion idea W08 · Day 1 · Event Ingestion at Scale
Numbers to remember

100 KB max event size
1,000 max events per batch request
7 · 30 · 90 days retention for raw, user and order topics; payments kept forever
90% queue usage that triggers load shedding

sdm w08 day-01-event-ingestion numbers W08 · Day 1 · Event Ingestion at Scale
Logs vs events

	Logs	Events
Format	Unstructured text	Structured, schema-aware
Parsing	Fragile regex	Self-describing
Aggregation	Hard	Easy
Schema change	Breaks parsers	Versioned evolution

sdm w08 day-01-event-ingestion tradeoff W08 · Day 1 · Event Ingestion at Scale
Classic mistakes

Events without an `event_id`: no deduplication, no tracing through the pipeline.
Writing to the database before replying: a slow database makes a slow API.
No dead letter queue, so a failed event is lost forever.

sdm w08 day-01-event-ingestion mistakes W08 · Day 1 · Event Ingestion at Scale
How do you handle offline phones? Buffer on the device and send in batches. A client-generated UUID deduplicates retried batches; keep both client and server timestamps; accept old events with a warning, reject ones too far in the future. sdm w08 day-01-event-ingestion ask W08 · Day 2 · Streaming vs Batch Processing
Two ways to process Batch processes large chunks: hours of latency, exact, cheaper, simpler. Streaming processes each event as it arrives: seconds of latency, approximate because late events may be missed, always on and more complex. sdm w08 day-02-streaming-vs-batch idea W08 · Day 2 · Streaming vs Batch Processing
Lambda Every event feeds a Spark batch layer (accurate, overwrites its views) and a Flink speed layer that fills the gap since the last batch; the serving layer adds them. Its big problem: two codebases that drift apart. sdm w08 day-02-streaming-vs-batch idea W08 · Day 2 · Streaming vs Batch Processing
Kappa One streaming codebase over a long-retention log. To reprocess, start a new version from the earliest offset, let it catch up, switch and delete the old output. 1B events at 100K/s replay in about 3 hours. sdm w08 day-02-streaming-vs-batch idea W08 · Day 2 · Streaming vs Batch Processing
Watermarks Watermark = max event time seen minus max out-of-orderness. With a 5-minute bound and latest event at 10:02 it's 9:57, so `[9:00-10:00)` fires only after an event at 10:05 or later. Wall-clock time plays no part. sdm w08 day-02-streaming-vs-batch idea W08 · Day 2 · Streaming vs Batch Processing
Late data and state An event after its window fired can be dropped, side-output, refire the window, or fit within allowed lateness. Keep large state in RocksDB with a TTL, and checkpoint so a crash doesn't start over. sdm w08 day-02-streaming-vs-batch idea W08 · Day 2 · Streaming vs Batch Processing
Write once, run both Apache Beam, Flink (batch is just bounded streaming) and Spark Structured Streaming run the same aggregation as a stream or a batch, so real-time and batch results can't drift apart. sdm w08 day-02-streaming-vs-batch idea W08 · Day 2 · Streaming vs Batch Processing
Numbers to remember

10,000 s to replay 1B events at 100K/s: about 3 hours
5 minutes max out-of-orderness in the watermark example
30 minutes of inactivity ends a user session
< 1,000 events of lag before a new Kappa version takes over

sdm w08 day-02-streaming-vs-batch numbers W08 · Day 2 · Streaming vs Batch Processing
Lambda vs Kappa

	Lambda	Kappa
Codebases	Two: Spark and Flink	One
Reprocessing	Batch recompute	Replay the log
Weak spot	Results diverge	Replay time grows with history
Choose when	PB history, batch ML, reconciliation	Real-time first, replay fast enough

sdm w08 day-02-streaming-vs-batch tradeoff W08 · Day 2 · Streaming vs Batch Processing
Classic mistakes

Writing batch output with `mode("append")`, so a retried job duplicates rows; overwrite the date partition.
Keeping state forever with no TTL, until it runs out of memory.
Grouping by a skewed key without salting, so one task runs for hours.

sdm w08 day-02-streaming-vs-batch mistakes W08 · Day 2 · Streaming vs Batch Processing
Why not stream everything? Late data: refunds, corrections and late mobile events need allowed lateness and retractions. Cost: streaming is always on; batch uses spot instances. Finance waits for all late data. Kappa works at manageable scale. sdm w08 day-02-streaming-vs-batch ask W08 · Day 3 · Data Modeling for Analytics
OLTP vs OLAP OLTP runs the business: many small transactions, normalized (3NF), row storage. OLAP understands it: few large queries over many rows and few columns, denormalized, column storage. "Revenue by country" becomes one scan and a GROUP BY. sdm w08 day-03-data-modeling idea W08 · Day 3 · Data Modeling for Analytics
Star schema A fact table of measures (revenue, quantity) at a stated grain, here one row per line item, with foreign keys to denormalized dimensions: product, customer, date, store. `order_id` is a degenerate dimension kept in the fact. sdm w08 day-03-data-modeling idea W08 · Day 3 · Data Modeling for Analytics
SCD Type 2 On a change, close the old row (`valid_to`, `is_current = FALSE`) and insert another. Facts store the surrogate key valid at sale time, so a plain join is point-in-time. Only tracked columns create versions. sdm w08 day-03-data-modeling idea W08 · Day 3 · Data Modeling for Analytics
Load dimensions, then facts Dimensions first, so today's new versions have surrogate keys. Lookups use the version valid on the batch date, not `is_current`, so backfills stay right. A missing product maps to unknown key -1, keeping totals whole. sdm w08 day-03-data-modeling idea W08 · Day 3 · Data Modeling for Analytics
Columnar storage `SUM(revenue)` over a 4-column table reads 1 column: 75% less I/O, better compression, vectorized. Parquet row groups carry statistics for predicate pushdown; Delta Lake and Iceberg add ACID transactions and time travel. sdm w08 day-03-data-modeling idea W08 · Day 3 · Data Modeling for Analytics
Partition and pre-aggregate Partition by filter columns like date, Z-order by a second filter such as `product_id`, never partition by a high-cardinality key. A daily rollup by category scans 365 × ~100 = 36,500 rows instead of 1B. sdm w08 day-03-data-modeling idea W08 · Day 3 · Data Modeling for Analytics
Numbers to remember

3.65 billion rows after adding country, segment and channel: 45-second queries
75% less I/O reading 1 of 4 columns
256MB–1GB target size per Parquet file
100MB–1GB target size per partition
> 10K partitions counts as over-partitioning

sdm w08 day-03-data-modeling numbers W08 · Day 3 · Data Modeling for Analytics
SCD types

	Type 1	Type 2	Type 3
On change	Overwrite	Add a row	Add a column
History	None	Full	One level
Cost	Simplest	More storage, complex queries	Simple current vs previous

sdm w08 day-03-data-modeling tradeoff W08 · Day 3 · Data Modeling for Analytics
Classic mistakes

An order-level grain: you can never drill down to products.
Business keys in the fact table instead of surrogate keys, so dimension history is lost.
Partitioning by `customer_id`: 10 million partitions full of small files.

sdm w08 day-03-data-modeling mistakes W08 · Day 3 · Data Modeling for Analytics
Dashboards join 5-6 tables and take 30 seconds. Fix? A star schema: line-item sales facts with denormalized dimensions removes most joins. Add rollups for common queries, and partition by date so "last 7 days" reads 7 partitions. sdm w08 day-03-data-modeling ask W08 · Day 4 · Late-Arriving Data and Correctness
Event time, not processing time A purchase made at 14:30 on the subway reaches the server at 15:45, 75 minutes late. Windowed by processing time it lands in 15:00-16:00; by event time it lands in 14:00-15:00, where it happened. sdm w08 day-04-late-data idea W08 · Day 4 · Late-Arriving Data and Correctness
Two separate knobs Watermark = max event time minus the out-of-orderness bound: with a 5-minute bound and max 10:03 it's 9:58, and an older event can't lower it. Allowed lateness is separate: how long a fired window still accepts updates. sdm w08 day-04-late-data idea W08 · Day 4 · Late-Arriving Data and Correctness
Per-key watermarks idle out With per-key watermarks the global one is the minimum, so a phone offline for two days pins it in the past and windows stop closing. Keys not seen within the idle timeout are left out. sdm w08 day-04-late-data idea W08 · Day 4 · Late-Arriving Data and Correctness
Tune lateness from data Measure the lateness distribution (example: p50 2 seconds, p99 5 minutes, max 72 hours) and set it per source: minutes for web, hours for mobile. Unlimited lateness grows state until it runs out of memory. sdm w08 day-04-late-data idea W08 · Day 4 · Late-Arriving Data and Correctness
Patch or reprocess Patch additive metrics (SUM, COUNT) when late data is small. Reprocess from raw events, overwriting the range, for non-additive ones (AVG, MEDIAN, DISTINCT), big late volumes or changed logic. sdm w08 day-04-late-data idea W08 · Day 4 · Late-Arriving Data and Correctness
Version, label, finalize Store every computation as a version with a reason and delta, so latest, final and as-of queries work. Stream results are provisional; a daily batch job rereads raw events for final numbers. Label dashboards Live or Final. sdm w08 day-04-late-data idea W08 · Day 4 · Late-Arriving Data and Correctness
Numbers to remember

75 minutes late: the subway purchase example
< 5% of original volume: late data small enough to patch
~99% · ~99.9% completeness of the T+1 preliminary and T+7 final reports
2 AM daily finalization job

sdm w08 day-04-late-data numbers W08 · Day 4 · Late-Arriving Data and Correctness
Patch vs reprocess

	Patch	Reprocess
Late volume	Small	Significant
Aggregations	SUM, COUNT	AVG, MEDIAN, DISTINCT
Cost	Quick	Expensive, exact
Writes	Apply a delta	Overwrite the range

sdm w08 day-04-late-data tradeoff W08 · Day 4 · Late-Arriving Data and Correctness
Classic mistakes

Windowing by processing time, so offline events land in the wrong hour.
Overwriting results in place, leaving no answer to "why did yesterday change?"
Alerting on live numbers; gate the normal threshold on `is_final`, a stricter one for live.

sdm w08 day-04-late-data mistakes W08 · Day 4 · Late-Arriving Data and Correctness
Daily revenue rises 5-10% the next day. Fix it. Measure lateness first. Use event-time watermarks and side-output anything beyond tolerance; version every result; label live vs final, with a T+1 job reprocessing the day; compare with "yesterday at this time". sdm w08 day-04-late-data ask W08 · Day 5 · Query Layer and Optimization
Match the engine to the access pattern Cloud warehouses (Snowflake, BigQuery) answer in seconds, managed, for ad-hoc SQL. Real-time OLAP (ClickHouse, Druid) gives milliseconds and high concurrency for user-facing dashboards. Trino queries the lake in place; DuckDB runs in-process. sdm w08 day-05-query-optimization idea W08 · Day 5 · Query Layer and Optimization
Why the dashboard is slow "Revenue by category, region and day for 90 days" scans ~2.7B of ~11B rows (30M a day), shuffles 50GB and takes 47 seconds: no partitioning, no pre-aggregation, no caching, joins that shuffle. sdm w08 day-05-query-optimization idea W08 · Day 5 · Query Layer and Optimization
Partition, then cluster Partitioning decides which files to read, and only helps when the query filters on the partition column. Clustering (sorting, Z-ordering) decides how rows sit inside files, so `category = 'A'` can skip files without it. sdm w08 day-05-query-optimization idea W08 · Day 5 · Query Layer and Optimization
Pre-aggregate the predictable Pre-aggregate patterns that are predictable and frequent (over 10 a day): the executive dashboard drops from 45 seconds to 50ms. Route to the coarsest rollup that keeps the needed granularity; leave rare ad-hoc slices on demand. sdm w08 day-05-query-optimization idea W08 · Day 5 · Query Layer and Optimization
Cache with the right TTL Layers: CDN, Redis query results (60-80% hits for popular queries), database cache, block cache. TTL follows freshness: about a minute for today, hours for history. Include the tenant in the key: same SQL, different rows. sdm w08 day-05-query-optimization idea W08 · Day 5 · Query Layer and Optimization
Isolate tenants' data and compute Row-level security is cheap, but one missing filter leaks data. Per-tenant quotas (10 concurrent queries, 100GB scanned a day, 300-second timeout) stop one noisy tenant from slowing everyone else. sdm w08 day-05-query-optimization idea W08 · Day 5 · Query Layer and Optimization
Numbers to remember

8.7M · 365K · 12K rows in hourly, daily, monthly rollups
8 grouping sets from a CUBE over 3 dimensions
50ms vs 45 s dashboard query on a rollup vs raw facts
10TB vs 10GB scanned without and with a partition filter

sdm w08 day-05-query-optimization numbers W08 · Day 5 · Query Layer and Optimization
Tenant isolation patterns

	Row-level security	Schema per tenant	Database per tenant
Isolation	One missing filter leaks	Strong	Complete
Main cost	Noisy neighbors	Schema proliferation	Highest cost, most ops
Cross-tenant queries	Easy	Harder	None

sdm w08 day-05-query-optimization tradeoff W08 · Day 5 · Query Layer and Optimization
Classic mistakes

Caching everything for 24 hours, so live panels show stale numbers.
One `agg_everything` rollup grouped by every dimension: nearly as big as the fact table.
No query timeouts or byte limits, so runaway queries block others and costs explode.

sdm w08 day-05-query-optimization mistakes W08 · Day 5 · Query Layer and Optimization
Queries cost $80K a month; 10 queries drive 70%. Plan? Fix those 10: rollups or materialized views (1TB scanned becomes 1GB), Redis caching (one scan, 49 hits), date partitioning and clustering. For the long tail: byte limits, timeouts, cost estimates. sdm w08 day-05-query-optimization ask W08 · Day 6 · Counting at Scale: Bloom Filters, HyperLogLog, Count-Min Sketch and Top-K
Counting means remembering Exact distinct counts cost memory per distinct key, per slice, per day: 10M visitors in 3 of 10,000 cells is ~1.2GB a day. Sketches cost a fixed amount with a known, bounded error. sdm w08 day-06-counting-at-scale idea W08 · Day 6 · Counting at Scale: Bloom Filters, HyperLogLog, Count-Min Sketch and Top-K
Bloom filter: "no" is certain Set k bits per key; any 0 bit means never inserted. False positives only: ~9.6 bits per key at 1%. Over-fill silently ruins it: 3x the planned keys pushes 1% to ~44%. sdm w08 day-06-counting-at-scale idea W08 · Day 6 · Counting at Scale: Bloom Filters, HyperLogLog, Count-Min Sketch and Top-K
HyperLogLog counts distinct Registers keep the longest run of leading zeros in hashed ids. Error is `1.04/√m`: 0.81% in 12KB at p = 14. Re-adding an id changes nothing, so replays don't inflate it. sdm w08 day-06-counting-at-scale idea W08 · Day 6 · Counting at Scale: Bloom Filters, HyperLogLog, Count-Min Sketch and Top-K
Merge sketches, never sum uniques HLLs merge by register-wise max into the sketch of the union. Store one per day and slice; any range is a merge. Summing daily uniques counts visitor-days, not visitors. sdm w08 day-06-counting-at-scale idea W08 · Day 6 · Counting at Scale: Bloom Filters, HyperLogLog, Count-Min Sketch and Top-K
Count-Min for heavy hitters only d rows of w counters; read the minimum. Never under-counts, over by at most `εN` of the whole stream. At ε = 0.001 over 50M views, a 2,000 estimate means anything from 0 to 2,000. sdm w08 day-06-counting-at-scale idea W08 · Day 6 · Counting at Scale: Bloom Filters, HyperLogLog, Count-Min Sketch and Top-K
Top-k with Space-Saving Keep m counters; a new key evicts the minimum and inherits it as its error, so counts over-estimate by at most N/m. Key by item, one summary per minute pane, merge the last 60. sdm w08 day-06-counting-at-scale idea W08 · Day 6 · Counting at Scale: Bloom Filters, HyperLogLog, Count-Min Sketch and Top-K
Numbers to remember

9.6 bits · k = 7 Bloom filter per key at 1% false positives
12KB · 0.81% HLL at precision 14 (Redis)
~53KB Count-Min at ε = 0.1%, δ = 1%
108GB vs ~11GB 90 days of exact sets vs daily HLL sketches

sdm w08 day-06-counting-at-scale numbers W08 · Day 6 · Counting at Scale: Bloom Filters, HyperLogLog, Count-Min Sketch and Top-K
Which sketch for which question

	Bloom	HyperLogLog	Count-Min	Space-Saving
Answers	Seen x?	How many distinct?	How often x?	Which x most?
Error	False "yes" only	Both ways	Over only	Over only
Replay-safe	Yes	Yes	No	No

sdm w08 day-06-counting-at-scale tradeoff W08 · Day 6 · Counting at Scale: Bloom Filters, HyperLogLog, Count-Min Sketch and Top-K
Classic mistakes

Dropping events a Bloom filter "has seen": 1% of new events silently lost. Verify "maybe" exactly.
Adding daily or regional unique counts instead of merging sketches.
Merging per-partition top-10 lists: an item 11th everywhere vanishes.

sdm w08 day-06-counting-at-scale mistakes W08 · Day 6 · Counting at Scale: Bloom Filters, HyperLogLog, Count-Min Sketch and Top-K
Unique visitors for any date range, per category and country? One HLL++ sketch per day per slice at precision 14, stored as a rollup column. Any range merges daily sketches, ±0.8%. Keep exact `COUNT(DISTINCT)` in batch for audited numbers. sdm w08 day-06-counting-at-scale ask W08 · Boss · Real-Time Analytics for Video Streaming Platform
Size the firehose 10M peak viewers sending one event every 10 seconds is 1M events a second; a day adds up to ~37B, planned as 50B. Heartbeats are 70% (35B). At ~300 bytes that's 15TB a day raw, 3TB compressed. sdm w08 capstone-interview-week-8-streaming-analytics idea W08 · Boss · Real-Time Analytics for Video Streaming Platform
Two serving stores Regional collectors write to Kafka (`events.playback` keyed by `user_id`, 7-day retention). Flink feeds ClickHouse for trending and live counts; Spark feeds BigQuery for history and A/B tests; raw events land in S3. sdm w08 capstone-interview-week-8-streaming-analytics idea W08 · Boss · Real-Time Analytics for Video Streaming Platform
Acknowledge after Kafka Collectors validate and batch, but answer the client only after Kafka acks: acking earlier would lose buffered events if a collector crashed. 20 collectors per region handle ~10K events a second each. sdm w08 capstone-interview-week-8-streaming-analytics idea W08 · Boss · Real-Time Analytics for Video Streaming Platform
Trending from unique viewers Raw counts favour long content and bots, so score unique viewers (HyperLogLog) in a 1-hour window sliding every minute, normalized by a category baseline and boosted by velocity. Scores refresh every 30-60 seconds. sdm w08 capstone-interview-week-8-streaming-analytics idea W08 · Boss · Real-Time Analytics for Video Streaming Platform
Three freshness tiers Events under 5 minutes late update real-time numbers; under 24 hours they queue for the T+1 preliminary job at 6 AM; later ones wait for the T+7 final job. Phones persist events in SQLite. sdm w08 capstone-interview-week-8-streaming-analytics idea W08 · Boss · Real-Time Analytics for Video Streaming Platform
Creator analytics A star schema (`fact_daily_views` with five dimensions) partitioned by viewing date and clustered by content. 90% of queries hit the `agg_daily_content` rollup; HLL sketches merge uniques across days instead of summing them. sdm w08 capstone-interview-week-8-streaming-analytics idea W08 · Boss · Real-Time Analytics for Video Streaming Platform
Numbers to remember

1M/s peak events: 10M viewers, one event every 10 seconds
270TB · 810TB 90 days of compressed raw events, before and with 3x replication
< 60 s trending freshness
80% cache hit-rate target on dashboards

sdm w08 capstone-interview-week-8-streaming-analytics numbers W08 · Boss · Real-Time Analytics for Video Streaming Platform
Freshness tiers

	Real-time	Preliminary	Final
When	Every minute	T+1, 6 AM	T+7
Late events	Under 5 minutes	Most late data	All, up to 7 days
Status	Provisional	Preliminary	Official, immutable

sdm w08 capstone-interview-week-8-streaming-analytics tradeoff W08 · Boss · Real-Time Analytics for Video Streaming Platform
Classic mistakes

Acknowledging clients before Kafka acks, so a collector crash loses events.
Counting raw events for trending, which favours long titles and bots.
Summing daily unique viewers over 30 days, so a 5-day viewer counts 5 times.

sdm w08 capstone-interview-week-8-streaming-analytics mistakes W08 · Boss · Real-Time Analytics for Video Streaming Platform
How does this scale to 10x? 200 collectors per region or Kafka/Kinesis as the first hop, 500 partitions per topic, 1,000 Flink workers, 100+ ClickHouse nodes. Sample heartbeats at 10%, tier storage, pre-aggregate more: 10x traffic for 3-4x cost. sdm w08 capstone-interview-week-8-streaming-analytics ask W09 · Day 1 · Tenant Isolation Strategies
The isolation spectrum Shared tables with a `tenant_id` column, then a schema per tenant, a database per tenant, dedicated infrastructure and separate cloud accounts. Each step adds isolation and cost; no tenant identifier at all is never acceptable. sdm w09 day-01-tenant-isolation idea W09 · Day 1 · Tenant Isolation Strategies
Tenant from the token The verified JWT claim is authoritative; a subdomain or `X-Tenant-ID` header is only a hint and a mismatch is rejected. The tenant lives in a `ContextVar`, so concurrent requests can't see each other's tenant. sdm w09 day-01-tenant-isolation idea W09 · Day 1 · Tenant Isolation Strategies
Fail loudly without a tenant `get_current_tenant()` raises `TenantContextError` when no context is set. Any fallback, a default tenant or the previous request's, is exactly how a query ends up without its tenant filter. sdm w09 day-01-tenant-isolation idea W09 · Day 1 · Tenant Isolation Strategies
RLS as defense in depth Application repositories add `tenant_id` to every query; PostgreSQL RLS filters again in the database. `FORCE` makes policies apply to the table owner too, and `app.current_tenant` is cleared before a pooled connection is released. sdm w09 day-01-tenant-isolation idea W09 · Day 1 · Tenant Isolation Strategies
Context must travel Events carry `tenant_id` (also the partition key) and jobs take it as an explicit argument, because a `ContextVar` doesn't reach another process. Cache keys, file paths, search indexes, logs and metrics all include the tenant. sdm w09 day-01-tenant-isolation idea W09 · Day 1 · Tenant Isolation Strategies
Numbers to remember

Millions · thousands · hundreds of tenants: shared tables, schema, database per tenant
500 × 10 = 5,000 connections for 500 tenant databases with 10-connection pools
10 s · 100 · 50,000 Salesforce limits per transaction: CPU, SOQL queries, records
5 minutes tenant context cache TTL

sdm w09 day-01-tenant-isolation numbers W09 · Day 1 · Tenant Isolation Strategies
Where tenants live

	Shared tables	Schema per tenant	Database per tenant
Isolation	Application	Database	Physical
Backup	All or nothing	Per schema	Per tenant
Data residency	Difficult	Possible	Easy
Enterprise ready	Rarely	Sometimes	Yes

sdm w09 day-01-tenant-isolation tradeoff W09 · Day 1 · Tenant Isolation Strategies
Classic mistakes

A query without `WHERE tenant_id = ?`, returning every tenant's rows.
Trusting a client-supplied `X-Tenant-ID`, so an attacker picks the victim's tenant.
Cache keys like `user:{user_id}` without a tenant prefix: IDs repeat across tenants.

sdm w09 day-01-tenant-isolation mistakes W09 · Day 1 · Tenant Isolation Strategies
Design a SaaS for 50,000 organizations, startups to enterprises. Tier it: shared PostgreSQL with `tenant_id` and RLS for standard customers, schema or dedicated databases for enterprise. Propagate tenant context from the JWT through every service, cache, queue and log. sdm w09 day-01-tenant-isolation ask W09 · Day 2 · Noisy Neighbor Prevention
One tenant breaks everyone One "export all data" report scanned 2.3 billion rows for 847 seconds: API latency went from 200ms to 15,000ms and 500 other tenants couldn't use the product. Legitimate use, no limits. sdm w09 day-02-noisy-neighbour idea W09 · Day 2 · Noisy Neighbor Prevention
Rate limits aren't enough Rate limiting stops request volume, not expensive operations: five requests a minute can still scan billions of rows. Add concurrency, compute and volume quotas, and enforce at the gateway, application, database and infrastructure. sdm w09 day-02-noisy-neighbour idea W09 · Day 2 · Noisy Neighbor Prevention
Limits shared across servers A token bucket refills up to `burst_size`: short bursts pass, the long-run rate holds. Per-server limiters give a tenant N times its limit, so one atomic Lua script updates a Redis sliding-window log. sdm w09 day-02-noisy-neighbour idea W09 · Day 2 · Noisy Neighbor Prevention
Guard the database Concurrency slots expire after their TTL, so a crashed worker can't leak one forever. EXPLAIN-based analysis rejects queries over 10 million estimated rows unless the plan is Enterprise; per-tenant pools set statement timeouts. sdm w09 day-02-noisy-neighbour idea W09 · Day 2 · Noisy Neighbor Prevention
Schedule fairly Round-robin across per-tenant queues: a tenant with 100 queued jobs can't make another wait for all of them, and each queue is capped. Weighted fair queuing (free 1, pro 5, enterprise 20) gives paying tiers a larger share. sdm w09 day-02-noisy-neighbour idea W09 · Day 2 · Noisy Neighbor Prevention
Numbers to remember

60 · 1,000 · 10,000 API calls a minute: free, pro, enterprise
5 · 50 · 200 concurrent requests by plan
30 · 120 · 300 s query timeout by plan
80% of a quota: warn before the hard limit

sdm w09 day-02-noisy-neighbour numbers W09 · Day 2 · Noisy Neighbor Prevention
Limit layers

	Blocks	Example
API gateway	Request volume	Per-tenant rate limit
Application	Expensive operations	Concurrency, query analysis
Database	Runaway statements	Timeouts, per-tenant pools
Infrastructure	Resource exhaustion	CPU and memory quotas

sdm w09 day-02-noisy-neighbour tradeoff W09 · Day 2 · Noisy Neighbor Prevention
Classic mistakes

One global limiter: a tenant at 900 req/sec leaves 100 for everyone else.
Throttling silently by sleeping, which hides the limit and holds connections; return 429 with Retry-After.
One limit for every operation: 1,000 exports melt the database.

sdm w09 day-02-noisy-neighbour mistakes W09 · Day 2 · Noisy Neighbor Prevention
How do you stop one customer affecting others? Per-tenant token buckets at the gateway, concurrency limits and query analysis in the app, per-tenant pools and statement timeouts in the database. When limits hit: 429, Retry-After, usage dashboards, alerts at 80%. sdm w09 day-02-noisy-neighbour ask W09 · Day 3 · Data Residency and GDPR
Four words to keep apart Residency is where data is stored; sovereignty is which country's laws apply; localization is a legal requirement to keep data in-country; a transfer is moving data across borders, which needs a legal basis. sdm w09 day-03-data-residency idea W09 · Day 3 · Data Residency and GDPR
What counts as personal data Names, emails, IP addresses, cookie and device IDs, location and behaviour. Pseudonymized data is still personal data, and backups and caches follow the same rules as the primary data. sdm w09 day-03-data-residency idea W09 · Day 3 · Data Residency and GDPR
Regional data, global control plane The common SaaS pattern: each tenant is assigned a region at onboarding, and its database, storage and search live there. The global control plane holds only metadata: tenant registry, configuration, routing rules. sdm w09 day-03-data-residency idea W09 · Day 3 · Data Residency and GDPR
Consent done right Freely given, specific per purpose, informed, unambiguous, and as easy to withdraw as to give. Keep an append-only log (exact text, version, IP, time) plus a current-status table; never asked counts as not granted. sdm w09 day-03-data-residency idea W09 · Day 3 · Data Residency and GDPR
Transfers need a basis Adequacy decision first, then Standard Contractual Clauses with supplementary measures, or BCRs for corporate groups. Post-Schrems II, document a Transfer Impact Assessment and review it, because adequacy can be revoked. sdm w09 day-03-data-residency idea W09 · Day 3 · Data Residency and GDPR
Numbers to remember

€20M or 4% of global turnover, whichever is higher: top GDPR fine
2020 · 2023 Schrems II ends Privacy Shield; EU-US Data Privacy Framework
1 hour tenant region cache TTL: invalidate on migration
365 days until a transfer assessment is reviewed

sdm w09 day-03-data-residency numbers W09 · Day 3 · Data Residency and GDPR
Residency patterns

	Single region	Regional deployments	Regional data, global control
Compliance	Can't meet residency	Strong	Strong
Cross-region features	All	None	Some
Fits	Non-regulated data	Strict localization	Most SaaS

sdm w09 day-03-data-residency tradeoff W09 · Day 3 · Data Residency and GDPR
Classic mistakes

Keeping EU data in Frankfurt but sending backups to a US bucket.
Logging emails or user IDs to a global logging service.
Using US vendors without a DPA and a transfer mechanism, or assuming adequacy is permanent.

sdm w09 day-03-data-residency mistakes W09 · Day 3 · Data Residency and GDPR
We need to serve EU customers. What changes? EU database, storage and search, with tenants assigned a region at onboarding; contract basis for the service, consent for marketing and analytics; regional or aggregated analytics; DPAs, a subprocessor list, audit logs. sdm w09 day-03-data-residency ask W09 · Day 4 · Right to Deletion
What Article 17 asks Respond within one month (two more for complex cases), delete every copy including backups eventually, and have processors delete too. You may refuse for legal obligations, public interest or legal claims, and must explain why. sdm w09 day-04-right-to-deletion idea W09 · Day 4 · Right to Deletion
Deleting is not marking Soft delete leaves the data in storage and backups, so it's only a step before hard delete. Pseudonymized data is still personal; truly anonymous aggregates ("users made 10,000 orders") may be kept. sdm w09 day-04-right-to-deletion idea W09 · Day 4 · Right to Deletion
Inventory, then orchestrate Map every place personal data lives, then run one executor per system in dependency order: search, files, analytics, third parties, the primary database (it holds the user_id mapping and foreign keys), and caches last. sdm w09 day-04-right-to-deletion idea W09 · Day 4 · Right to Deletion
Stop, retry, verify Targets at the same priority run in parallel; after a failed group the run stops and a retry resumes at that point. The request completes only after every system is re-checked and the data is confirmed gone. sdm w09 day-04-right-to-deletion idea W09 · Day 4 · Right to Deletion
Keep what the law requires Orders are financial records, so they're anonymized (user_id NULL, addresses 'DELETED'), not deleted. Shared content shows "Deleted User". The deletion audit trail is kept, with the user ID redacted, as proof. sdm w09 day-04-right-to-deletion idea W09 · Day 4 · Right to Deletion
Backups expire, restores re-delete You can't remove one user from a backup, so backups expire under a retention policy that's tracked. If a backup is restored first, the pending deletions are re-applied before the data goes live. sdm w09 day-04-right-to-deletion idea W09 · Day 4 · Right to Deletion
Numbers to remember

1 month (+2) to respond to an erasure request
30 days deletion request deadline in the service
k > 10 typical k-anonymity for aggregates you may keep
7 years tax retention for anonymized transaction records

sdm w09 day-04-right-to-deletion numbers W09 · Day 4 · Right to Deletion
Ways to "delete"

	Hard delete	Anonymize	Pseudonymize
Data afterwards	Gone	Can't identify anyone	Still personal data
Satisfies erasure	Yes	Yes, if truly anonymous	No
Use for	Most personal data	Financial records, stats	Protecting active data

sdm w09 day-04-right-to-deletion tradeoff W09 · Day 4 · Right to Deletion
Classic mistakes

Deleting the users row and forgetting search, analytics, caches and third parties.
Reporting "deleted" without a verification pass over every system.
Deleting audit logs with the user, leaving no proof of what happened.

sdm w09 day-04-right-to-deletion mistakes W09 · Day 4 · Right to Deletion
Walk me through a full deletion request. Start a 30-day timer, run executors in dependency order, anonymize orders kept for tax law, verify every system, then log, confirm and report. Backups purge on expiry; restores re-apply pending deletions. sdm w09 day-04-right-to-deletion ask W09 · Day 5 · Security Architecture
Defense in depth Five layers, each assuming the others might fail: perimeter (WAF, DDoS protection, rate limits), network (VPC, private subnets), application (authn, authz, validation), data (encryption, keys), monitoring (audit logs, anomaly detection). sdm w09 day-05-security-architecture idea W09 · Day 5 · Security Architecture
Zero trust Never trust, always verify: network location grants nothing, every connection is authenticated and every request authorized. Least privilege, assume breach, micro-segmentation. Google built BeyondCorp after Operation Aurora. sdm w09 day-05-security-architecture idea W09 · Day 5 · Security Architecture
Secrets never in code Not in code, config files, container images or environment variables. Fetch from Vault or Secrets Manager at connection time so rotation needs no restart. A leaked secret is already copied everywhere: rotate it. sdm w09 day-05-security-architecture idea W09 · Day 5 · Security Architecture
Encrypt at every layer TLS 1.2+ externally and mTLS between services; TDE and SSE-KMS at rest; field-level encryption for SSNs and cards, because TDE decrypts for any query. A key hierarchy (HSM master, tenant keys, data keys) limits blast radius. sdm w09 day-05-security-architecture idea W09 · Day 5 · Security Architecture
Revocable tokens 15-minute access JWTs and 7-day refresh tokens, plus a server-side session checked on every request, so deleting the session revokes access at once. Five failed logins mean a 15-minute lockout; errors stay generic. sdm w09 day-05-security-architecture idea W09 · Day 5 · Security Architecture
Numbers to remember

15 minutes · 7 days access and refresh token lifetimes
5 attempts · 15 minutes failed logins before lockout, and its length
12+ bcrypt cost factor
5 minutes secrets cache TTL

sdm w09 day-05-security-architecture numbers W09 · Day 5 · Security Architecture
STRIDE

	Threat	Mitigation
Spoofing	Pretending to be someone	MFA, certificates
Tampering	Modifying data or code	Validation, signing
Repudiation	Denying an action	Audit logs
Disclosure, DoS, privilege	Leaks, outages, escalation	Encryption, rate limits, least privilege

sdm w09 day-05-security-architecture tradeoff W09 · Day 5 · Security Architecture
Classic mistakes

Trusting frontend validation: `amount=-500` sent straight to the API adds money.
Fetching documents by ID alone, without `AND tenant_id = ?` from the authenticated user.
`allow_origins=["*"]` with credentials, or logging passwords, card numbers and auth headers.

sdm w09 day-05-security-architecture mistakes W09 · Day 5 · Security Architecture
How would you secure a multi-tenant SaaS? WAF and TLS 1.2+ at the edge, private subnets; tenant from the JWT, enforced in app and RLS; bcrypt, short JWTs, admin MFA; secrets in Vault; audit logs, a runbook. sdm w09 day-05-security-architecture ask W09 · Boss · Design a Multi-Tenant Enterprise SaaS Platform
Size it first ~100M documents at 500KB is 50TB; with metadata 60TB, 180TB with 3x replication. Extracted text of 10KB each is a 1TB index, ~3TB in Elasticsearch. 1,000 requests a second at peak, 60% of them searches. sdm w09 capstone-interview-week-9-multi-tenant-saas idea W09 · Boss · Design a Multi-Tenant Enterprise SaaS Platform
Regional data planes, global control plane US, EU and APAC each run a full stack: API, PostgreSQL, Elasticsearch, S3, Redis, Kafka. The global control plane holds only the tenant registry, flags, quotas and routing. No tenant data crosses regions. sdm w09 capstone-interview-week-9-multi-tenant-saas idea W09 · Boss · Design a Multi-Tenant Enterprise SaaS Platform
Five isolation layers Tenant from the verified JWT (the subdomain must match); repositories that add `tenant_id`; RLS in PostgreSQL; a search index per enterprise tenant or a filtered shared index; S3 paths and IAM by tenant, with per-tenant keys. sdm w09 capstone-interview-week-9-multi-tenant-saas idea W09 · Boss · Design a Multi-Tenant Enterprise SaaS Platform
Quotas and fair queues Per-plan quotas on API calls, searches, uploads and query timeouts. Each tenant gets its own capped job queue; dividing the plan weight by `(1 + depth * 0.1)` shrinks a deep queue's share. sdm w09 capstone-interview-week-9-multi-tenant-saas idea W09 · Boss · Design a Multi-Tenant Enterprise SaaS Platform
Uploads go straight to S3 The upload service issues a presigned URL for the tenant's regional bucket, so bytes skip the API servers; text extraction and thumbnails run asynchronously from a queue. The server picks the bucket, never the client. sdm w09 capstone-interview-week-9-multi-tenant-saas idea W09 · Boss · Design a Multi-Tenant Enterprise SaaS Platform
Numbers to remember

500ms · 200ms p99 targets for search and API calls
< 4 hours · < 1 hour RTO and RPO
7 days lifetime of a GDPR export download link
30 days offboarding grace period before full deletion

sdm w09 capstone-interview-week-9-multi-tenant-saas numbers W09 · Boss · Design a Multi-Tenant Enterprise SaaS Platform
Standard vs enterprise tenants

	Standard	Enterprise
Database	Shared tables with RLS	Dedicated database
Search	Shared index, `tenant_id` filter	Index per tenant
Isolation level	SHARED (RLS)	DATABASE, own KMS key
Index corruption hits	Every tenant on the index	One tenant

sdm w09 capstone-interview-week-9-multi-tenant-saas tradeoff W09 · Boss · Design a Multi-Tenant Enterprise SaaS Platform
Classic mistakes

Reading `tenant_id` from the request body so admins can "switch tenants".
Putting user emails in the global control plane for faster login routing.
Deleting the primary database first, leaving search, file and analytics copies you can no longer find.

sdm w09 capstone-interview-week-9-multi-tenant-saas mistakes W09 · Boss · Design a Multi-Tenant Enterprise SaaS Platform
How would this scale from 500 to 5,000 tenants? Auto-scale API servers and workers; shard PostgreSQL by `tenant_id` so a tenant stays together, with dedicated shards for big ones; per-tenant search indexes for large tenants; new regions like Japan. sdm w09 capstone-interview-week-9-multi-tenant-saas ask W10 · Day 1 · SLIs, SLOs and SLAs
Measure, target, promise An SLI is the measurement (share of requests under 200ms), an SLO the internal target (99.9% over 30 days), an SLA the promise with consequences (99.5%, or a 10% credit). SLO stricter than SLA. sdm w10 day-01-slis-slos-slas idea W10 · Day 1 · SLIs, SLOs and SLAs
Specify the SLI precisely Good events over valid events, measured at the load balancer for the client's view. Exclude health checks and planned maintenance; count 4xx as good, because the service handled them correctly. sdm w10 day-01-slis-slos-slas idea W10 · Day 1 · SLIs, SLOs and SLAs
Appropriate reliability 100% is impossible, infinitely expensive and leaves no room to ship. Going from 99.9% to 99.99% may cost ~10x infrastructure and slow features, yet most users can't tell. Don't set the SLO at current performance: leave headroom. sdm w10 day-01-slis-slos-slas idea W10 · Day 1 · SLIs, SLOs and SLAs
Error budgets drive decisions 99.9% over 30 days allows 43.2 minutes of failure. Above 50% left, ship freely; 20-50%, no risky deploys; under 20%, pause features; exhausted, freeze. The policy must be written down and enforced. sdm w10 day-01-slis-slos-slas idea W10 · Day 1 · SLIs, SLOs and SLAs
Alert on burn rate Burn rate is the actual error rate over the allowed one; 1x empties the budget exactly at the window's end. Page only when a short and a long window both exceed it, so blips don't page. sdm w10 day-01-slis-slos-slas idea W10 · Day 1 · SLIs, SLOs and SLAs
Numbers to remember

8.76 h · 52.6 min · 5.26 min downtime a year at 99.9%, 99.99%, 99.999%
43.2 minutes error budget for 99.9% over 30 days
72 hours notice for planned maintenance excluded from SLOs
3-5 SLOs per service

sdm w10 day-01-slis-slos-slas numbers W10 · Day 1 · SLIs, SLOs and SLAs
Burn-rate alerts

	Burn rate	Budget used	Action
1 hour	14.4x	2%	Page immediately
6 hours	6x	5%	Page in business hours
1 day	3x	10%	Ticket
3 days	1x	10%	Weekly review

sdm w10 day-01-slis-slos-slas tradeoff W10 · Day 1 · SLIs, SLOs and SLAs
Classic mistakes

SLOs on server metrics like CPU < 70%: requests can fail at 50% CPU.
Setting the SLO at current performance, so normal variation keeps breaching it.
Fifty SLOs instead of 3-5 user-facing ones, or SLOs with no error budget policy.

sdm w10 day-01-slis-slos-slas mistakes W10 · Day 1 · SLIs, SLOs and SLAs
How do you know your notification system is reliable? SLIs for availability, acceptance latency, delivery latency and delivery rate; targets 99.9%, p99 < 200ms, 95% within 30 seconds, 99.5% delivered. An error budget policy decides velocity; burn-rate alerts page. sdm w10 day-01-slis-slos-slas ask W10 · Day 2 · Observability
Monitoring vs observability Monitoring watches for known failure modes with pre-defined dashboards and alerts: it tells you when. Observability lets you ask questions you didn't anticipate, like slicing slow requests by app version: it tells you why. You need both. sdm w10 day-02-observability idea W10 · Day 2 · Observability
Three pillars, one trace ID Metrics say when and how bad, logs what happened, traces where in the request path. A trace ID propagated through every service and written in every log, plus exemplars on metrics, turns hours into minutes. sdm w10 day-02-observability idea W10 · Day 2 · Observability
Pick the right metric type Counters only go up (use `rate`), gauges go up and down, histograms count observations in buckets. Histograms add up across pods for a fleet-wide p99; summaries pre-compute per-pod quantiles that can't be aggregated. sdm w10 day-02-observability idea W10 · Day 2 · Observability
Keep labels bounded Total series is the product of label cardinalities: method × endpoint × status is 10,000 series, but adding `user_id` for 1M users makes 10 billion. Per-user detail belongs in logs and traces. sdm w10 day-02-observability idea W10 · Day 2 · Observability
RED for services, USE for resources Rate, errors and duration for APIs; utilization, saturation and errors for databases, queues and disks. Dashboards show the four golden signals first and compare with yesterday and last week. sdm w10 day-02-observability idea W10 · Day 2 · Observability
Propagate the context Service B continues A's trace only if A injects the context (`traceparent` or B3 headers) and B extracts it. Otherwise each service starts its own trace and the journey is lost. sdm w10 day-02-observability idea W10 · Day 2 · Observability
Numbers to remember

5 × 100 × 20 = 10,000 series for method, endpoint, status
5% for 5 minutes 5xx rate that pages critical, with a runbook
80% for 10 minutes DB connection pool: a warning, not a page
15 minutes unresolved before paging senior on-call

sdm w10 day-02-observability numbers W10 · Day 2 · Observability
The three pillars

	Metrics	Logs	Traces
Cardinality	Low	High	Medium
Cost	Cheap	Medium	High
Best for	Alerts, trends	Debugging, audit	Latency breakdown

sdm w10 day-02-observability tradeoff W10 · Day 2 · Observability
Classic mistakes

Alerting on causes like CPU > 80% instead of symptoms users feel, such as p99 latency.
Logs without context ("Payment failed") or with secrets: no passwords, full cards or tokens.
Alerts without runbooks, or `error_count > 0`, which is pure noise.

sdm w10 day-02-observability mistakes W10 · Day 2 · Observability
Paged for high latency. Now what? Dashboard: p99 2.3s on `/api/orders`, errors normal. A slow trace shows a 2-second products query; its logs show 500 rows, where p99 was 10. Add a limit now, alert on row counts. sdm w10 day-02-observability ask W10 · Day 3 · Deployment Strategies
Small and frequent The safest deployment is a small one; the safest rollback is a fast one. Deploy daily in small increments, early in the week, watch dashboards during the deploy, and declare success only after the bake time. sdm w10 day-03-deployment-strategies idea W10 · Day 3 · Deployment Strategies
Rolling and blue-green Rolling replaces a few instances at a time: no downtime, but slow rollback, and both versions share the database and APIs. Blue-green switches all traffic at once; rollback is instant because the old side still runs. sdm w10 day-03-deployment-strategies idea W10 · Day 3 · Deployment Strategies
Canary with automated analysis Shift 1%, 10%, 50%, then 100%, watching each stage for 15 minutes. Roll back if errors rise over 1% absolute or p99 over 10% versus stable; under 100 requests, fail closed, never promote. sdm w10 day-03-deployment-strategies idea W10 · Day 3 · Deployment Strategies
Decouple deploy from release Ship code with the flag off, then enable it gradually. Hashing `flag_key:user_id` into a bucket 0-99 keeps each user on one path. Remove release flags after rollout; keep kill switches. sdm w10 day-03-deployment-strategies idea W10 · Day 3 · Deployment Strategies
Expand, migrate, contract To rename a column: add the new one and write both; backfill, then switch reads; drop the old column only when no running code uses it. Old and new code run together during every deploy. sdm w10 day-03-deployment-strategies idea W10 · Day 3 · Deployment Strategies
Numbers to remember

1% · 10% · 50% · 100% canary stages, 15 minutes each
1% · 10% canary limits: absolute error rise, relative p99 rise
5% · 1 s · 5 minutes auto-rollback error rate, p99 and watch window
2x infrastructure for blue-green

sdm w10 day-03-deployment-strategies numbers W10 · Day 3 · Deployment Strategies
Deployment strategies

	Rollback	Cost	Risk exposure
Rolling	Slow	1x + buffer	Gradual
Blue-green	Instant	2x	All users at once
Canary	Fast	1x + small	Limited users first
Feature flag	Instant	1x	Chosen users

sdm w10 day-03-deployment-strategies tradeoff W10 · Day 3 · Deployment Strategies
Classic mistakes

Renaming a column in place while old and new code both run.
Schema and code changes in one deploy, so neither can roll back alone.
No rollback plan, or a deploy on Friday followed by walking away.

sdm w10 day-03-deployment-strategies mistakes W10 · Day 3 · Deployment Strategies
How would you deploy a new shipping-cost calculation? Create the new table first, deploy code behind an off flag, shadow-run both calculations and compare, then enable for internal users and 1% of customers. A bug means flipping the flag off. sdm w10 day-03-deployment-strategies ask W10 · Day 4 · Capacity Planning
The capacity cycle Measure what you can handle now, forecast demand (growth, seasonality, events), find what breaks first, plan the scaling, validate with load tests, then measure again. Scale before you need to: projects take 4-6 weeks. sdm w10 day-04-capacity-planning idea W10 · Day 4 · Capacity Planning
Headroom Headroom = (max capacity − current) / max capacity. A database good for 3,000 QPS at 2,000 has 33%. Keep at least 30% on critical resources: traffic is bursty, retries storm, GC pauses happen, and 100% means already failing. sdm w10 day-04-capacity-planning idea W10 · Day 4 · Capacity Planning
Capacity is the sustainable max Stress-test the latency curve: in the example, latency stays within the SLO up to 1,000 RPS and errors start at 1,800. Capacity is 1,000, not the breaking point; run at most ~700. sdm w10 day-04-capacity-planning idea W10 · Day 4 · Capacity Planning
Little's Law L = λ × W. 500 RPS at 200ms needs 100 concurrent workers: 25 pods at 4 workers each, ~36 with 30% headroom. Past capacity, requests queue, W grows, L grows, and latency spirals. sdm w10 day-04-capacity-planning idea W10 · Day 4 · Capacity Planning
Stay below 70-80% In a single-server queue, wait ≈ service time × ρ / (1 − ρ): about 1x at 50% utilization, 4x at 80%, 9x at 90%, infinite at 100%. Never run critical resources above 70-80%. sdm w10 day-04-capacity-planning idea W10 · Day 4 · Capacity Planning
Pre-scale known events Auto-scaling takes 3-5 minutes, so a sudden spike lands first. Pre-scale for launches and campaigns, scale up fast and down slowly (+3 pods vs −1 with a 15-minute cooldown), and plan for peaks, not averages. sdm w10 day-04-capacity-planning idea W10 · Day 4 · Capacity Planning
Numbers to remember

≥ 30% headroom on critical resources
4-6 weeks lead time for a scaling project
3-5 minutes auto-scaling lag
5x Black Friday traffic multiplier in the forecast

sdm w10 day-04-capacity-planning numbers W10 · Day 4 · Capacity Planning
Scale up vs scale out

	Vertical	Horizontal
Limit	Hardware ceiling	Nearly unlimited
Failure	Single point of failure	None
Complexity	Simple	Distributed, stateless
Use for	Primary databases, quick fixes	Stateless services, workers

sdm w10 day-04-capacity-planning tradeoff W10 · Day 4 · Capacity Planning
Classic mistakes

Waiting for problems: by the time you see them, users are affected.
Trusting averages: 30% average CPU can hide an evening peak near 95%.
Load testing only in staging, or ignoring the database and third-party limits behind you.

sdm w10 day-04-capacity-planning mistakes W10 · Day 4 · Capacity Planning
How do you size for a dinner rush? 7,000 orders a minute × ~100 API calls is ~12,000 RPS; at 100ms, 1,200 concurrent: 60 pods of 20 workers, 90 with headroom. For 20x events, test at 25x and pre-scale. sdm w10 day-04-capacity-planning ask W10 · Day 5 · Incident Management
Severity sets the response SEV1, complete outage over $10,000 an hour: all hands, execs, status page. SEV2, a major degradation: on-call plus backup. SEV3 waits for business hours; SEV4 goes to the ticket queue. Not every alert is an incident. sdm w10 day-05-incident-management idea W10 · Day 5 · Incident Management
Mitigate first Detect, triage, respond, mitigate, resolve, learn. Stop user impact before chasing the root cause: recent deploy, roll back; dependency down, fail over or degrade; overloaded, scale up or rate limit. sdm w10 day-05-incident-management idea W10 · Day 5 · Incident Management
Clear roles The Incident Commander coordinates and decides but doesn't debug; the Technical Lead diagnoses and fixes; the Communications Lead runs updates and the timeline. One change at a time, announced first. sdm w10 day-05-incident-management idea W10 · Day 5 · Incident Management
Communicate on a clock First status page update within 10 minutes; SEV1 updates at least every 15 minutes, even "still investigating". Plain, impact-focused language; never blame a provider you chose. sdm w10 day-05-incident-management idea W10 · Day 5 · Incident Management
Blameless postmortems Within 48 hours. Ask why the system allowed it, never stop at human error, and use the five whys until you reach something fixable. Action items get owners and dates and are tracked like any work. sdm w10 day-05-incident-management idea W10 · Day 5 · Incident Management
Sustainable on-call At least 8 people in rotation, fewer than 5 pages per week-long shift, handoff meetings between shifts, and only actionable alerts with runbooks. 50 noisy pages a week train people to ignore the real one. sdm w10 day-05-incident-management idea W10 · Day 5 · Incident Management
Numbers to remember

< 5 min · < 15 min · < 1 hour SEV1 targets to detect, engage, mitigate
10 · 15 minutes first status update, then SEV1 cadence
48 hours to hold the postmortem
≥ 8 people · < 5 pages on-call rotation and weekly page budget

sdm w10 day-05-incident-management numbers W10 · Day 5 · Incident Management
Blame vs blameless

	Blame	Blameless
Asks	Who did it?	How did the process allow it?
People	Hide mistakes	Report early, share near-misses
Fix	"Be more careful"	Automated gates
Outcome	Repeat incidents	Prevented incidents

sdm w10 day-05-incident-management tradeoff W10 · Day 5 · Incident Management
Classic mistakes

Too many cooks: several people changing things at once.
Going silent while heads down, so customers learn from Twitter.
No postmortem, or one that stops at "the engineer forgot to test".

sdm w10 day-05-incident-management mistakes W10 · Day 5 · Incident Management
Walk me through a major incident. Acknowledge and assess, declare severity, open a channel, assign IC, tech and comms leads. Mitigate first: rollback, failover or scaling. Update every 15 minutes, then a blameless postmortem within 48 hours with tracked actions. sdm w10 day-05-incident-management ask W10 · Boss · The Grand Finale
Size the bidding platform 10 billion requests a day is ~116,000 a second, ~500,000 at peak. At ~8 KB of audit data each: 80 TB a day, 29 PB a year, ~200 PB over 7 years, so tiered storage. sdm w10 capstone-interview-week-10-the-grand-finale idea W10 · Boss · The Grand Finale
A 100ms latency budget Receive 2ms, parse 5ms, DSP fan-out 75ms, auction 10ms, respond 3ms: 95ms, with 5ms to spare. The 70ms DSP timeout is a hard deadline inside the fan-out; a late answer loses the impression. sdm w10 capstone-interview-week-10-the-grand-finale idea W10 · Boss · The Grand Finale
Budget leases, never overspend A global coordinator splits each budget across regions, keeping a reserve; regions lease small amounts to pods with an atomic Redis check-and-decrement. Pods spend locally, so no network call per bid; no lease means no bid. sdm w10 capstone-interview-week-10-the-grand-finale idea W10 · Boss · The Grand Finale
Fail toward under-spending If a region fails, its leases expire after 30 seconds and only provably unspent budget is released; the rest stays locked until reconciliation. In a partition a region bids on existing leases, then stops. sdm w10 capstone-interview-week-10-the-grand-finale idea W10 · Boss · The Grand Finale
Fan out with breakers Query 5-10 DSPs in parallel under one global deadline and auction whatever arrived. A per-DSP breaker opens after 5 consecutive failures and retries after 30 seconds. Under overload, query the top DSPs and shed low-value requests. sdm w10 capstone-interview-week-10-the-grand-finale idea W10 · Boss · The Grand Finale
Audit through an outbox Insert the bid event into an outbox before replying (one local write); Debezium ships it to Kafka keyed by `advertiser_id` for ordered billing. Hot 7 days, warm Parquet to 90 days, Glacier to 7 years. sdm w10 capstone-interview-week-10-the-grand-finale idea W10 · Boss · The Grand Finale
Numbers to remember

70ms DSP timeout inside a 75ms fan-out window
0.1% maximum budget overspend allowed
10 million · 1 million slow and failed requests a day the SLOs allow
> 0.95 · 0.5-0.95 fraud score that blocks, and that flags and bids lower

sdm w10 capstone-interview-week-10-the-grand-finale numbers W10 · Boss · The Grand Finale
Consistency by component

	Budget	Auction	Audit
Needs	Strong	Eventual	Durable, ordered
Why	Overspend is a contract breach	Stale data is harmless	Billing and 7-year compliance
How	Atomic leases	Local decisions	Outbox to Kafka

sdm w10 capstone-interview-week-10-the-grand-finale tradeoff W10 · Boss · The Grand Finale
Classic mistakes

Each server checking "is there budget?" alone: $10 left becomes $19,990 overspent.
Publishing audit events straight to Kafka beside the budget write: a dual write can lose them.
Waiting on slow DSPs instead of one hard deadline and circuit breakers.

sdm w10 capstone-interview-week-10-the-grand-finale mistakes W10 · Boss · The Grand Finale
Cut latency to 50ms. Where first? DSP fan-out, 75ms of the budget: query 5 DSPs instead of 10, predict which will bid and skip the rest, cut the timeout from 70ms to 50ms, cache DSP bid models at the edge. sdm w10 capstone-interview-week-10-the-grand-finale ask W11 · Day 1 · LLM Serving: Tokens, Latency and Cost
Size it in tokens, not requests A model call's work, price and rate limits scale with tokens in and out. 900 turns a minute sounds small; at ~3,835 input tokens a turn it's 3.45M input tokens a minute. sdm w11 day-01-llm-serving idea W11 · Day 1 · LLM Serving: Tokens, Latency and Cost
Prefill, then decode Prefill reads every input token in one parallel, compute-bound pass and sets time to first token. Decode writes one token per step, bound by memory bandwidth, and sets time per output token. sdm w11 day-01-llm-serving idea W11 · Day 1 · LLM Serving: Tokens, Latency and Cost
Output length is the latency lever Latency = TTFT + TPOT × output tokens. A 250-token reply at 20 ms a token is ~5 s of decode. Halving output may cut ~50% of latency; halving the prompt only 1-5%. sdm w11 day-01-llm-serving idea W11 · Day 1 · LLM Serving: Tokens, Latency and Cost
Tokens resident set concurrency The KV cache holds every token of every running sequence in GPU memory. A 20,000-token prompt holds the memory of five 4,000-token ones, so long prompts cut concurrency and raise TTFT. sdm w11 day-01-llm-serving idea W11 · Day 1 · LLM Serving: Tokens, Latency and Cost
Errors arrive after the 200 Stream end to end: no buffering proxies, SSE comment heartbeats under idle timeouts, cancel on disconnect. An `overloaded_error` can arrive mid-stream: tell the client in-band, store the reply as `interrupted`. sdm w11 day-01-llm-serving idea W11 · Day 1 · LLM Serving: Tokens, Latency and Cost
Admit by tokens in a gateway One gateway counts tokens, keeps shared buckets for input and output, reserves a share for interactive chat, queues with deadlines, and retries 429 and 529 only before the first byte, with `retry-after` and jitter. sdm w11 day-01-llm-serving idea W11 · Day 1 · LLM Serving: Tokens, Latency and Cost
Numbers to remember

23,010 · 1,500 input and output tokens per 6-turn conversation
~$0.061 a conversation on Sonnet 5.5, input ≈ 75% (October 2026 prices)
3.45M vs 2M input tokens a minute at peak vs Start-tier ITPM
5.2× input for 3.3× the turns

sdm w11 day-01-llm-serving numbers W11 · Day 1 · LLM Serving: Tokens, Latency and Cost
Vendor API vs self-hosted

	Vendor API	Self-hosted (vLLM)
Cost shape	Per token, no idle cost	GPU-hours, busy or idle
Capacity	Rate limits, overload errors	What you provisioned
Data	Leaves your network	Stays in your network

sdm w11 day-01-llm-serving tradeoff W11 · Day 1 · LLM Serving: Tokens, Latency and Cost
Classic mistakes

Sizing by requests: 900 a minute is 3.45M input tokens a minute.
Treating 200 as success, or saving a partial reply as the answer.
Retrying 429s at once, after the first byte, or on a spend-cap 429.

sdm w11 day-01-llm-serving mistakes W11 · Day 1 · LLM Serving: Tokens, Latency and Cost
Why is the first token slow at peak? TTFT includes queueing and prefill. At peak, requests wait for KV memory on the provider's GPUs. Shorter prompts, a cached prefix and a reserved interactive lane for chat help. sdm w11 day-01-llm-serving ask W11 · Day 2 · Embeddings and Vector Search
Keywords miss meaning, vectors miss tokens "paisa wapas" needs an embedding to find the refunds article; "RTN-4471" needs BM25, because one return code embeds much like another. Use the same embedding model and version for queries and documents. sdm w11 day-02-embeddings-and-vector-search idea W11 · Day 2 · Embeddings and Vector Search
Exact search is a bandwidth problem 3M vectors × 4 KB is 12 GB per query, ~0.6 s at 20 GB/s. Keep exact search for ground truth and selective filters: one customer's 2,000 tickets scan faster than any index. sdm w11 day-02-embeddings-and-vector-search idea W11 · Day 2 · Embeddings and Vector Search
HNSW, a graph you descend Sparse top layers make long greedy jumps; layer 0 holds every vector and keeps a candidate list of size `ef`. Raising `ef_search` buys recall with latency. Deleted rows linger until a vacuum or rebuild. sdm w11 day-02-embeddings-and-vector-search idea W11 · Day 2 · Embeddings and Vector Search
IVF and product quantisation IVF probes `nprobe` of `nlist` k-means clusters; cheap appends, but centroids drift. PQ stores a 1,024-dim vector as 64 one-byte codes, 64× smaller; rescore the top 100-200 with full vectors. sdm w11 day-02-embeddings-and-vector-search idea W11 · Day 2 · Embeddings and Vector Search
Filter inside the search pgvector post-filters approximate scans: `ef_search` 40 at 10% selectivity returns about 4 rows, silently. Use iterative scans, partial indexes for a few known values, or an exact scan for tiny sets. sdm w11 day-02-embeddings-and-vector-search idea W11 · Day 2 · Embeddings and Vector Search
Fuse ranks, not scores BM25 is unbounded and query-dependent; cosine sits in a narrow band. Reciprocal rank fusion sums 1 ÷ (60 + rank) per retriever, so agreement wins. Pass the reranker a generous candidate list. sdm w11 day-02-embeddings-and-vector-search idea W11 · Day 2 · Embeddings and Vector Search
Numbers to remember

4,224 B per 1,024-dim vector in HNSW, M = 16 (4,096 + 128)
~13.5 GB · ~422 GB HNSW float32 at 3.2M and at 100M vectors
142 B · 1,040 B 1-bit BBQ and int8 per 1,024-dim vector (Elastic)
k = 60 the RRF constant

sdm w11 day-02-embeddings-and-vector-search numbers W11 · Day 2 · Embeddings and Vector Search
HNSW vs IVF-PQ

	HNSW	IVF-PQ
Recall	Very high, tuned by ef	Lower; rescoring recovers most
Memory per vector	~4,224 B	~72 B, floats on disk
Updates	Incremental, deletes are tombstones	Cheap appends, retrain as data drifts

sdm w11 day-02-embeddings-and-vector-search tradeoff W11 · Day 2 · Embeddings and Vector Search
Classic mistakes

Vector search alone, so order codes and SKUs go missing.
Filtering after the index and getting 3 results instead of 10, with no error.
Querying with a new embedding model against vectors from the old one.

sdm w11 day-02-embeddings-and-vector-search mistakes W11 · Day 2 · Embeddings and Vector Search
Why not HNSW everywhere? At 3.2M vectors I would: ~13.5 GB fits one node. At 100M it's ~422 GB of RAM per replica for 15 queries a second, a memory problem, so quantise or IVF-PQ with rescoring from SSD. sdm w11 day-02-embeddings-and-vector-search ask W11 · Day 3 · RAG Systems
The answer is only as fresh as the index RAG is a search pipeline with a model at the end. Lewis et al. swapped 2016 and 2018 indexes and accuracy collapsed; MegaMart's stale policy was a hot-swap that never happened. sdm w11 day-03-rag-systems idea W11 · Day 3 · RAG Systems
Chunk along structure Parse tables with their headings, chunk one section at a time with document and section titles inside, and dedup by content hash. Contextual retrieval prepends 50-100 tokens of document context to each chunk. sdm w11 day-03-rag-systems idea W11 · Day 3 · RAG Systems
Freshness is an SLO CDC keyed by `doc_id`, idempotent upserts on `doc_id:version:chunk_no`, deletes as events, a dead-letter queue for poison documents, a reconciliation sweep, and a canary measuring publish to searchable. sdm w11 day-03-rag-systems idea W11 · Day 3 · RAG Systems
Retrieve wide, rerank narrow Hybrid search returns ~150 candidates; a cross-encoder reads query and chunk together and keeps ~8. Below a score threshold, abstain and offer a human rather than answer from weak context. sdm w11 day-03-rag-systems idea W11 · Day 3 · RAG Systems
Retrieval runs as the user Filter by tenant and ACL inside retrieval, so unauthorised chunks never reach the reranker, the model or the logs. A prompt saying "don't reveal" isn't access control. Entitlements down: public content only. sdm w11 day-03-rag-systems idea W11 · Day 3 · RAG Systems
Numbers to remember

~980M tokens · ~$59 to embed the corpus (voyage-4, October 2026)
~2 hours full re-embed at 8M tokens a minute
5.7% → 1.9% top-20 failures with contextual retrieval plus reranking
< 200,000 tokens knowledge base small enough for a cached prompt

sdm w11 day-03-rag-systems numbers W11 · Day 3 · RAG Systems
RAG or something else?

	RAG	Long context	Tool call	Fine-tuning
Fits	Large, changing, permissioned	Small, stable	Live records	Behaviour, not facts
Example	Policy corpus	60-page FAQ	Order status	House style

sdm w11 day-03-rag-systems tradeoff W11 · Day 3 · RAG Systems
Classic mistakes

Tuning the prompt before reading the retrieved chunks.
Fixed-size chunks that cut tables from their headings.
Permissions in the prompt, or a filter applied after the top-k.

sdm w11 day-03-rag-systems mistakes W11 · Day 3 · RAG Systems
The reranker is timing out. What now? Don't silently send the top 8 unreranked: answers get quietly worse. Add a timeout and breaker, fall back to a smaller cross-encoder or a higher abstain threshold, and alert on unreranked answers. sdm w11 day-03-rag-systems ask W11 · Day 4 · Evaluation and Guardrails
Quality is measured, not unit-tested Outputs vary between runs, every prompt change is global, and wrong answers look like successes. Gate changes on a versioned golden set with never-regress cases, scored per slice of intent and language. sdm w11 day-04-evaluation-and-guardrails idea W11 · Day 4 · Evaluation and Guardrails
Graders, cheapest first Code checks facts, citations and behaviour. An LLM judge handles faithfulness and preference. Humans check the judge: accept it only if its kappa with people approaches the people's kappa with each other. sdm w11 day-04-evaluation-and-guardrails idea W11 · Day 4 · Evaluation and Guardrails
Judges are biased instruments Position, verbosity and self-preference. Compare in both orders and count a win only if it survives the swap; judge with a different model; give a reference; use binary, checkable rubrics. sdm w11 day-04-evaluation-and-guardrails idea W11 · Day 4 · Evaluation and Guardrails
Faithful isn't correct Score retrieval and generation separately. Smaller chunks dropped context recall from 0.91 to 0.78 while faithfulness rose: answers stuck to thinner context. The stale 30-day policy scored perfect faithfulness too. sdm w11 day-04-evaluation-and-guardrails idea W11 · Day 4 · Evaluation and Guardrails
Assume the model will be fooled Injection arrives in seller policies, uploads and tool results, and no filter catches it reliably. So limit what a fooled model can do: narrow tools, server-side checks, confirmation for side effects. sdm w11 day-04-evaluation-and-guardrails idea W11 · Day 4 · Evaluation and Guardrails
Break the lethal trifecta Private data plus untrusted content plus an outbound channel can leak data. The first two are the product, so remove the third: allow-listed links and images, `email_return_label(order_id)` instead of `send_email`. sdm w11 day-04-evaluation-and-guardrails idea W11 · Day 4 · Evaluation and Guardrails
Numbers to remember

12,000 calls · ~40 min one full eval run of 2,000 cases
~84% of the eval bill spent on judges
±1.6 points resolution of an 85% pass rate on 2,000 cases
~17 days thumbs A/B at 10% of traffic per arm

sdm w11 day-04-evaluation-and-guardrails numbers W11 · Day 4 · Evaluation and Guardrails
Who grades the answers

	Code	LLM judge	Humans
Good for	Facts, citations, behaviour	Faithfulness, preference	Ground truth
Weakness	Blind to meaning	Biased, needs calibration	Slow, disagree

sdm w11 day-04-evaluation-and-guardrails tradeoff W11 · Day 4 · Evaluation and Guardrails
Classic mistakes

Shipping on a vibe check of 15 hand-picked chats.
A classifier as the injection defence while the model picks amounts and recipients.
A fallback model nobody ran against the golden set.

sdm w11 day-04-evaluation-and-guardrails mistakes W11 · Day 4 · Evaluation and Guardrails
The injection screen is down. Now what? Switch to answers without actions: policy answers continue, side-effect tools are off, refunds go to agents. Never skip the layer, and don't block everything. Fail closed for actions, soft for information. sdm w11 day-04-evaluation-and-guardrails ask W11 · Day 5 · Cost, Caching and Operations
Four meters Fresh input, cache write, cache read and output, each priced per model. History is resent every turn, so cost grows with the square of the turns. Compare models on cost per resolved conversation. sdm w11 day-05-cost-caching-and-operations idea W11 · Day 5 · Cost, Caching and Operations
Prompt caching is cost and capacity An exact prefix, built tools, system, then messages. A 5-minute write is 1.25× input, a read 0.1× or less, and for most Claude models reads don't count toward ITPM. Stable first, volatile last. sdm w11 day-05-cost-caching-and-operations idea W11 · Day 5 · Cost, Caching and Operations
Semantic caches skip the guardrails A false hit is a confident wrong answer that never met the model or Day 4's checks. Only general, non-personal questions; keys scoped by tenant, language and prompt version; purge from sources; measure false hits. sdm w11 day-05-cost-caching-and-operations idea W11 · Day 5 · Cost, Caching and Operations
Route easy turns to a small model A router decides before the call, a cascade after scoring the cheap answer. At $0.0012 vs $0.022 a conversation, a cascade pays until ~95% escalate. Keep each conversation on one model. sdm w11 day-05-cost-caching-and-operations idea W11 · Day 5 · Cost, Caching and Operations
Budgets that degrade Reserve the worst case from `max_tokens`, settle from `usage`, per feature, tenant, user and conversation. At 80%, thrifty; at 95%, minimal; spent, hand off. A step cap per turn stops loops first. sdm w11 day-05-cost-caching-and-operations idea W11 · Day 5 · Cost, Caching and Operations
Fail over in steps One retry layer, in the gateway, with SDK retries off. The fallback's cache is cold, so a conversation costs ~3× until it warms. Shift 10%, 25%, 50% to avoid its acceleration limits. sdm w11 day-05-cost-caching-and-operations idea W11 · Day 5 · Cost, Caching and Operations
Numbers to remember

$0.061 · $0.022 · $0.0096 per conversation: no cache, cached, cached + routing
~$110,000 → ~$17,200 a month at 1.8M conversations
3.45M → ~290,000 counted input tokens a minute at peak, with caching
50% Batch API discount, for work nobody waits for

sdm w11 day-05-cost-caching-and-operations numbers W11 · Day 5 · Cost, Caching and Operations
Three caches

	Prompt cache	Exact answer cache	Semantic cache
Skips	Reprocessing a prefix	The model call	The model, for similar questions
Risk	None	Low	False, stale, leaked answers
Use for	Prompts, tools, history	Machine-made prompts	General policy questions

sdm w11 day-05-cost-caching-and-operations tradeoff W11 · Day 5 · Cost, Caching and Operations
Classic mistakes

A timestamp on line one of the system prompt, so every request misses the cache.
Charting Anthropic's `input_tokens` alone, which excludes cached tokens.
One organisation-wide budget, so a runaway loop takes the chat down.

sdm w11 day-05-cost-caching-and-operations mistakes W11 · Day 5 · Cost, Caching and Operations
The cache hit rate dropped overnight. Why? The prefix changed: something volatile added, tools reordered, or a model switch. Split cache-read share by prompt version, diff the prompts, and roll back; then watch 429s, because ITPM headroom fell too. sdm w11 day-05-cost-caching-and-operations ask W11 · Boss · Design an AI Customer-Support Assistant
The model proposes, code decides The model returns `tool_use` blocks; a deterministic controller runs the agent loop, checks every call in a tool gate, and decides. Ownership, amounts, limits, confirmation and idempotency live outside the model. sdm w11 capstone-interview-week-11-ai-support-assistant idea W11 · Boss · Design an AI Customer-Support Assistant
The safest argument doesn't exist `issue_refund` takes only an order id: the gate computes the amount with `refund_policy.allowed_amount()`, over ₹5,000 hands off. `email_return_label(order_id)` lets the server pick the recipient. No free text leaves. sdm w11 capstone-interview-week-11-ai-support-assistant idea W11 · Boss · Design an AI Customer-Support Assistant
Proposals, confirmed and executed once A side effect becomes a proposal row the customer confirms with a button showing the server's action. Confirming re-checks it and executes under an idempotency key from the proposal id, so double taps refund once. sdm w11 capstone-interview-week-11-ai-support-assistant idea W11 · Boss · Design an AI Customer-Support Assistant
Untrusted text only in tool results Seller policies arrive inside `lookup_order`'s result, JSON-encoded and labelled. The turn is then tainted, so any side effect needs confirmation. Where text sits lowers the odds; the gate decides safety. sdm w11 capstone-interview-week-11-ai-support-assistant idea W11 · Boss · Design an AI Customer-Support Assistant
A step cap, then instructive errors Five tool calls per turn, then an `is_error` result telling the model to offer a human; hand off if it keeps going. Errors say what to do next, not just what failed. sdm w11 capstone-interview-week-11-ai-support-assistant idea W11 · Boss · Design an AI Customer-Support Assistant
Keep the tools array identical For answers without actions, don't remove tools: they're the first cached segment, so every request would miss and hit 429s. The gate denies side effects instead, and pending proposals fail closed. sdm w11 capstone-interview-week-11-ai-support-assistant idea W11 · Boss · Design an AI Customer-Support Assistant
Numbers to remember

286 tokens tool-use system prompt, Sonnet 5.5 and Haiku 5.5, `tool_choice` auto (October 2026)
5 tool calls per turn before the step cap
10 minutes before an unconfirmed proposal expires
Zero target for side effects without a confirmed proposal

sdm w11 capstone-interview-week-11-ai-support-assistant numbers W11 · Boss · Design an AI Customer-Support Assistant
Where retrieved text enters

	Controller retrieves first	Model asks through a tool
Calls	One, first words ~1 s	An extra round trip
Used for	MegaMart's own policies	Seller policies, own tickets
Lands in	Newest turn, labelled sources	A `tool_result`

sdm w11 capstone-interview-week-11-ai-support-assistant tradeoff W11 · Boss · Design an AI Customer-Support Assistant
Classic mistakes

`issue_refund(order_id, amount)` executed as soon as the model calls it.
Removing tools from the request during an incident, breaking the cached prefix.
Letting a continuation replay a side effect instead of the confirm endpoint.

sdm w11 capstone-interview-week-11-ai-support-assistant mistakes W11 · Boss · Design an AI Customer-Support Assistant
Why not let the model call refunds directly? It reads customer and seller text and can't reliably tell data from instructions. So it proposes; code checks ownership, amount, limits and confirmation. A model bug costs a sentence, never a refund. sdm w11 capstone-interview-week-11-ai-support-assistant ask