Why it matters
GPTQ enables 4-bit LLMs. Understanding shapes quantization strategy.
Advertisement
The architecture
Per-layer quantization.
Hessian approximation.
Cholesky-based error correction.
Advertisement
How it works end to end
Frantar 2023.
Requires calibration data.
W4A16 (4-bit weights, 16-bit activations).
Group-wise scaling.