Why it matters

Prefill dominates TTFT for long prompts. Understanding math shapes latency budgets.

Advertisement

The architecture

Attention O(n^2 · d).

FFN O(n · d^2).

Compute-bound at large n.

Prefill mathAttentionO(n² · d)FFNO(n · d²)Bottleneckcompute-boundPrefill FLOPs grow quadratically with prompt length under standard attention
Prefill math.
Advertisement

How it works end to end

Quadratic scale with tokens.

Flash attention linearizes memory.

TTFT drops with faster compute.