Why it matters
BW-bound ops waste GPU. Understanding shapes optimization.
Advertisement
The architecture
Compare: FLOPs / bytes = arithmetic intensity.
Below balance: BW-bound.
Advertisement
How it works end to end
H100 balance: ~300 FLOP/byte for bf16 tensor.
Attention softmax: BW-bound.
LayerNorm, GELU: BW-bound.
Fuse into matmul for compute-bound.