LLM Serving Stacks — vLLM, TGI, TensorRT-LLM, SGLang

Why it matters

Serving stack choice affects cost + latency. Understanding shapes deployment.

The architecture

vLLM: PagedAttention + continuous batching + wide model support.

TGI: HuggingFace Text Generation Inference.

How it works end to end

vLLM: default choice for most. PagedAttention + continuous batching.

TensorRT-LLM: fastest on NVIDIA; complex conversion.

SGLang: primitives for structured generation, prefix reuse.

TGI: HuggingFace-integrated; solid.