Why it matters

Sub-second latency unlocks video apps that HLS/DASH can't do. Understanding enables new products.

Advertisement

The architecture

Codecs: VP8, VP9, H.264, AV1 (browser-dependent).

RTP over UDP: unreliable transport. Handles packet loss with FEC and NACK.

WebRTC video topologyPeer-to-peerdirect RTP over UDPSFUserver forwardsMCUserver mixesSFU is standard for multi-party; forwards streams without transcoding
Video topologies.
Advertisement

How it works end to end

SFU (Selective Forwarding Unit): server that receives from each participant and forwards to others. Doesn't decode; scales well.

MCU (Multipoint Control Unit): server that decodes, mixes, re-encodes. Higher CPU but reduces client bandwidth.

Adaptive bitrate: WebRTC dynamically adjusts based on bandwidth and CPU.

Simulcast: send multiple bitrates; SFU picks best for each viewer.