GPTQ
Frantar et al 2022. Layer-wise second-order quantization. High quality. ~4x smaller than FP16. Reference PTQ method.
Advertisement
AWQ
Activation-Aware Weight Quantization (Lin et al 2023). Protects salient weights (high-activation channels). Slightly better than GPTQ typically.
Advertisement
Format
Both compatible with vLLM, Transformers, TensorRT-LLM. AutoGPTQ + AutoAWQ libraries for conversion.