Why it matters
GQA is default modern LLM attention. Understanding shapes design.
Advertisement
The architecture
Q: H heads.
KV: G groups (G < H).
H/G Q heads share one KV.
Advertisement
How it works end to end
KV cache size: 1/(H/G) of MHA.
Quality: near-MHA for G ≥ 8.
Attention: replicate KV to Q shape or use special kernel.