Why it matters

FT scaling differs from pretraining. Understanding shapes strategy.

Advertisement

The architecture

SFT: quality data > quantity.

1k-10k high-quality often enough.

DPO: preference pairs scale differently.

FT scaling patternsSFT: 1k-10k qualitycuratedDPO: preference pairs10k-100k+Full FT: more computeless data-optimalLIMA showed quality > quantity in SFT; alignment tax varies
FT scaling.
Advertisement

How it works end to end

LIMA paper: 1k examples for good alignment.

Alpaca / OpenAssistant: larger + noisier.

DPO scales somewhat with data.