Disaggregated LLM Serving

Why it matters

Disaggregated architectures shape modern LLM serving. Understanding shapes production design.

The architecture

Prefill: compute-bound + variable duration.

Decode: memory-bound + steady tokens/sec.

Separate pools.

How it works end to end

DistServe / Splitwise / MoonCake designs.

Different SLOs per pool.

KV cache handed off between pools.