Why it matters

BitFit is minimal PEFT. Understanding shapes theory boundaries.

Advertisement

The architecture

Freeze all weights.

Train only bias parameters.

Tiny param count.

BitFit structureFreeze weightsno trainTrain biases onlytiny fractionSurprisingly effectivesome tasksZaken et al. 2022; effective for smaller tasks + models
BitFit.
Advertisement

How it works end to end

Zaken et al. 2022.

<0.1% params trained.

Effective for simpler tasks.

Baseline for PEFT comparisons.