KV Cache Compression

Why it matters

KV compression stretches context. Long-context apps need it to fit in HBM.

The architecture

Techniques: low-rank, top-k retention, sink tokens, sliding window.

Trade quality for capacity.

How it works end to end

H2O keeps heavy hitters.

StreamingLLM adds sinks.

ScissorHands cluster+drop.