Why it matters

AWQ often beats GPTQ. Understanding shapes quantization choice.

Advertisement

The architecture

Analyze activation magnitudes.

Scale salient channels.

Quantize.

AWQ flowActivation statsmagnitudeScale salientsprotect key channelsQuantize4-bitLin et al. 2023; often preferred over GPTQ for llama-family
AWQ.
Advertisement

How it works end to end

Salient identification via activation stats.

Scale factor per group.

W4A16 kernels.

vLLM + TRT-LLM support.