Prefill vs Decode Split

Why it matters

Prefill/decode split shapes serving. Understanding shapes optimization.

The architecture

Prefill: parallel over input tokens.

Decode: one token at a time.

Different scaling.

How it works end to end

TTFT (time to first token): prefill dominates.

ITL (inter-token latency): decode dominates.

Chunked prefill mixes.