Why it matters

PagedAttention scales serving. Understanding shapes memory model.

Advertisement

The architecture

KV cache: fixed-size pages.

Block table maps logical → physical.

Share prefixes across requests.

PagedAttention memory modelLogical KVper-request viewBlock tabletranslationPhysical pagesshared memoryAnalog of virtual memory: pages + page tables for KV cache
PagedAttention.
Advertisement

How it works end to end

Block size: e.g., 16 tokens.

Prefix sharing: multiple requests share pages.

Copy-on-write when diverging.