Compute-Bound Operations

Why it matters

Compute-bound ops are the goal. Understanding shapes optimization.

The architecture

Large matmul: arithmetic intensity high.

Attention QK^T + AV: compute-bound.

How it works end to end

Tensor cores: bf16/fp8 accelerate.

Batch dim > 128 typical for tensor core efficiency.

Kernel launch overhead: cudaGraphs to hide.