Paged KV Cache Deep Dive

Why it matters

Paged KV enables high throughput serving. Contiguous allocation wastes memory badly.

The architecture

Block table maps logical to physical.

Fixed block size.

Enables sharing.

How it works end to end

vLLM PagedAttention.

Block table indirection.

Prefix sharing across requests.