Why it matters

SmoothQuant enables activation quantization. Understanding shapes W8A8.

Advertisement

The architecture

Compute per-channel scale.

Divide activations, multiply weights.

Both quantize better.

SmoothQuant flowChannel scale sfrom statsDivide activationssmooth outliersMultiply weightscompensateXiao et al. 2022; W8A8 with quality; combine with per-tensor + per-channel
SmoothQuant.
Advertisement

How it works end to end

Alpha parameter tunes.

W8A8: both weights + activations int8.

Faster than W-only quant.