Core concept
V1 (2022), V2 (2023, 2x faster), V3 (2024, Hopper-optimized). Foundational for transformer inference.
Advertisement
How it works
Hardware-specific kernels. Compile per architecture. Some sequence patterns unsupported.
Advertisement
Trade-offs + gotchas
Every modern transformer. PyTorch integrated. vLLM + TRT-LLM depend.