Why it matters

Pruning is one of the levers for shrinking models. Not always applied (distillation often dominates), but critical when squeezing every byte for edge deployment.

Advertisement

The architecture

Magnitude pruning: remove weights with smallest absolute values. Simple, effective.

Structured pruning: remove entire attention heads or MLP channels. Enables hardware speedup.

Pruning workflowMagnitude prunesmallest weightsStructured pruneheads / channelsFine-tunerecover qualityPost-pruning fine-tuning usually needed to recover lost quality
Pruning cycle.
Advertisement

How it works end to end

2:4 structured sparsity: NVIDIA hardware supports 2 zeros in every 4 weights, giving 2x speedup. SLMs can be trained or pruned to this pattern.

N:M sparsity generalization: N zeros in every M consecutive weights.

Iterative pruning: prune a bit, fine-tune, repeat. Higher final sparsity than one-shot.