Why architecture matters here

Video encoder architecture matters because encoder choices determine every downstream cost. Bitrate = bandwidth. Encode speed = infrastructure. Quality = user perception. Right encoder + settings for the workload matters.

Cost is both encode compute and delivery bandwidth. Better codec cuts bandwidth 30-50% but costs more to encode.

Reliability of encoder output is about consistent quality. Rate control keeps within budget.

Advertisement

The architecture: every stage explained

Walk the diagram top to bottom.

Raw frames. YUV 4:2:0 typical; huge in raw form (1080p60 = 2 Gbps).

Encoder. H.264 (AVC), H.265 (HEVC), AV1 (open), VVC (next-gen). Each successive codec ~30-50% smaller at same quality.

Compressed bitstream. Encoded output; ~1-10 Mbps typical for streaming.

Motion estimation. Predict current frame from prior; encode difference (residual). Bulk of compression gain.

Transform + quantize. DCT (Discrete Cosine Transform) → quantize (drop high frequencies). Lossy step.

Rate control. CBR (constant bitrate), VBR (variable), CRF (constant quality). Encoder adjusts quantization to hit budget.

GOP structure. I frame (independent), P (predicts from prior), B (predicts from prior + future). GOP length affects seek + quality.

Entropy coding. CABAC (context-adaptive binary arithmetic) or CAVLC. Final compression step.

Hardware encode. NVIDIA NVENC, Apple VideoToolbox, Intel Quick Sync. 10-100x faster than SW but slightly lower quality.

Codec comparison. H.264 universal + fast. H.265 better + license fees. AV1 best + slower encode. VVC even better + patents.

Raw framesYUVEncoderH.264 / H.265 / AV1Compressed bitstreamsmallerMotion estimationprediction from prior framesTransform + quantizeDCT + QRate controltarget bitrate / VBR / CRFGOP structureI / P / B framesEntropy codingCABAC / CAVLCHardware encodeNVENC / VideoToolbox / QSVCodec comparisonAV1 best but slow to encodex264/x265/SVT-AV1 SW; NVENC + AMD + Intel HW encoders
Video encoder architecture: raw frames → motion estimation → transform + quantize → entropy coding → compressed bitstream; rate control + GOP structure + hardware.
Advertisement

End-to-end encode flow

Trace an encode. 1080p30 source. Target H.264 at 5 Mbps VBR.

Encoder analyzes GOP: 2-second GOP = 60 frames. 1 I frame, 5 P, 54 B.

First frame: I. Full intra encoding. Transform + quantize + entropy code. Result: ~200 KB.

Next frames: P/B. Motion estimation searches prior frame for similar blocks. Encodes motion vectors + residuals.

Rate control: monitor cumulative bitrate. If ahead of target, reduce quantization (better quality); if behind, increase (worse quality).

Output bitstream: ~5 Mbps average, VBR peaks 10 Mbps in high-motion scenes.

NVENC HW variant: same pipeline in silicon; 100x faster; quality slightly below x264 medium.

AV1 encode via SVT-AV1: ~30% smaller output at same quality; 10x slower to encode.