Capstone: analytics for a streaming platform
A full interview on real-time analytics for a video streaming service: clarifying questions, a million events a second and a petabyte of data, the Kafka, Flink, Spark, ClickHouse and BigQuery architecture, the event envelope and regional ingestion, a trending score from unique viewers, late data from planes and subways with real-time, preliminary and final numbers, a star schema for creators, query layers, scaling 10 times, edge cases and monitoring.
01 · Previously
W08 · BOSS · Real-Time Analytics for Video Streaming Platform
Six days of analytics. One interview.
Previously: counting at scale
Previously, on Day 6, you counted at scale: Bloom filters, HyperLogLog, Count-Min Sketch and top-k, each in fixed memory with a known error.
0:00 / 19:40