Triton — Python DSL for GPU Kernels

Why it matters

Triton democratizes GPU kernel writing. Understanding enables custom optimizations.

The architecture

Python decorators: @triton.jit marks kernel.

Kernels operate on tiles of data.

How it works end to end

Block-level ops: matmul, load, store on tiles. Higher level than CUDA thread.

Auto-tuning: try different tile sizes at runtime.

Integration: called from PyTorch or JAX.

Used in FlashAttention, xFormers, torch.compile kernels.