Why architecture matters here
Latency in live video is a product requirement wearing an infrastructure costume. Sports lose viewers to the neighbor's cheer and the push notification; auctions and betting are simply impossible beyond a few seconds (the bid window closes before the viewer sees it open); interactive formats — live commerce, Q&A, watch-together — die when the host answers a question the audience asked thirty seconds ago in their timeline. The architecture matters because latency is additive across the whole chain: encoder look-ahead + segmentation + origin + CDN buffering + player buffer, and no single stage can rescue a pipeline where another stage hoards seconds. Getting to 3 seconds is a systems discipline — every stage audited, every default distrusted.
The physics trade is unavoidable and must be engineered, not wished away: buffer is armor. An 18-second buffer absorbs an 8-second CDN hiccup invisibly; a 1.5-second buffer turns the same hiccup into a stall. Low-latency design therefore spends engineering everywhere else to compensate — faster segment availability (chunks), better bandwidth estimation (chunked-aware ABR), quicker recovery (smaller GOPs for faster quality switches), and rate-nudging so small disturbances bend playback speed instead of breaking playback. The honest framing for product: pick the latency tier the use case needs — 2–5s (LL-HLS/DASH), sub-second (WebRTC), or classic 15–30s (maximum robustness/scale) — because each tier has a different cost and failure profile, and 'as low as possible' is not a requirement.
Scale economics complete the argument. HTTP low-latency keeps CDN leverage: chunks are cacheable objects fanned out to millions of viewers with origin load measured in requests per segment, not per viewer. WebRTC's sub-second tier abandons that leverage — every viewer holds a stateful connection to an SFU fleet you scale per-viewer. The 2–5s HTTP tier exists precisely because it captures most of the product value at CDN prices; knowing where your use case sits on that curve is the architecture decision.
The architecture: every piece explained
Top row: making media available sooner. Capture and encode set the floor: low-latency encoder profiles cut look-ahead and B-frame depth (each B-frame is future-dependency = delay), tune rate control for steady output, and shrink GOPs (1–2s keyframe intervals) so quality switches and joins don't wait long for an IDR. Chunked CMAF is the packaging revolution: each 2–4s segment is written as a sequence of independently-flushable fragments (moof+mdat pairs, 200ms–1s each); the packager pushes each chunk to the origin the moment it exists, so the network pipeline carries media that is milliseconds old. LL-HLS (Apple's spec) exposes this as partial segments: the playlist advertises parts as they appear, clients hold blocking playlist requests (the server responds the instant the next part is ready — no polling gap), and preload hints let players request the next part before it's announced. LL-DASH achieves the same with chunked transfer encoding on an open segment plus manifest availability-time offsets — the player requests a segment still being written and receives chunks as they flush.
Middle row: the delivery and consumption halves. CDN behavior is the make-or-break: edges must forward chunked responses as chunks arrive (no store-and-forward of whole objects), honor blocking-request semantics, and collapse concurrent requests for the same part; a CDN that buffers full segments silently re-adds the seconds you removed — validating chunk pass-through per provider/config is deployment step one. Player tuning: target buffer 1–2s, join-at-live-edge logic, and ABR adapted for chunked reads — classic throughput estimation (bytes/time per segment download) breaks when downloads are paced by the encoder (you receive at media rate, not link rate), so LL players estimate from burst windows within chunks or use buffer-trajectory signals. Latency vs rebuffer is managed as an explicit control loop, and drift control implements it: playback-rate adjustment (±5% is imperceptible) nudges each viewer toward the target latency — falling behind → 1.05× catch-up; buffer thinning → 0.95× and a slightly higher latency setpoint. WebRTC occupies the sub-second tier: SFU-distributed, congestion-controlled per viewer, ~300–800ms glass-to-glass — the right tool for true interactivity and the wrong default for a million passive viewers.
Bottom rows: fitting product to physics. Use-case tiers: sports and news at 3–5s LL-HLS/DASH (scale + notification parity); auctions/betting at 1–3s aggressive LL or WebRTC; live commerce and watch-together sub-second WebRTC for participants with an LL-HLS tier for the audience — hybrid topologies are the norm, one truth (the encoder) feeding two stacks. The ops strip: glass-to-glass measured continuously (embedded timestamps read by instrumented players — not lab-only), origin scaling for blocking-request concurrency (thousands of held connections), and fallback ladders — when LL delivery degrades, players step to standard-latency profiles rather than stalling.
End-to-end flow
Follow a goal through a tuned pipeline. The camera feed hits the encoder (340ms: capture + encode at zero B-frames, 2s GOP); the packager cuts 2s segments into 500ms CMAF chunks, pushing each to origin as it closes (+120ms for the in-flight chunk). LL-HLS playlists advertise the new part; two hundred thousand players' blocking requests release simultaneously; edges — validated for chunk pass-through — relay the part with ~180ms added; players hold 1.4s buffers and render. Glass to glass: 2.7 seconds median, measured by the SEI-embedded timestamps the player SDK reports. The push notification (which rides a 2s data-feed path) now arrives roughly with the goal, not before it — the product metric the whole architecture serves.
Mid-match, reality tests the control loops. A regional transit blip slows edge fetches for 4 seconds: viewers' buffers thin from 1.4s toward 0.5s; players respond by dropping playback to 0.97× and letting latency drift to 3.5s rather than stalling; ABR — using burst-window estimation because segment-timing is encoder-paced — steps one rung down for two segments, then recovers. Rebuffer ratio for the event: 0.3%, at a latency classic streaming triples. One CDN POP misbehaves after a config rollout — full-segment buffering reappears, and that POP's viewers drift to 8s latency. The per-POP glass-to-glass dashboard catches the outlier in minutes (this is why latency is measured per edge, not globally); traffic steers away while the provider reverts.
The same event's auction-style overlay (predict-the-next-play with 5-second windows) runs the hybrid: participants on WebRTC (620ms median through the SFU tier), spectators on the LL-HLS feed with window-close timestamps carried in-band so the UI disables bids honestly per viewer's actual latency. The postmortem after the season quantifies the architecture's business case: notification-parity complaints down 90%, engagement on interactive overlays up, CDN egress costs unchanged (chunked CMAF didn't change byte volume — only timing), and the one standing action item is origin capacity: blocking playlist requests held 240k concurrent connections at peak, and the origin tier — not bandwidth — is what needed the next scaling round. Low latency moved the bottleneck from bytes to held connections, exactly as the design review predicted.