Why it matters

Activation quant unlocks full W8A8 / W4A4. Understanding shapes low-bit inference.

Advertisement

The architecture

Dynamic per-token/tensor scale.

Or offline calibrated static scale.

Outlier handling critical.

Activation quant flowActivation statsper token / channelScale online / offlinedynamic vs staticQuantizeint8 or int4Dynamic per-token scale often works; static requires good calibration
Activation quant.
Advertisement

How it works end to end

Per-token dynamic scale: robust.

Static scale: fast but calibration-sensitive.

SmoothQuant helps.