Why it matters

KTO enables single-label alignment. Easier data collection.

Advertisement

The architecture

Single label (good/bad).

Loss inspired by prospect theory.

No pairs needed.

KTO mathSingle labelup or downLossprospect theoryNo pairseasier collectionEthayarajh et al. 2024; simpler labels; competitive with DPO
KTO math.
Advertisement

How it works end to end

Ethayarajh 2024.

Single labels.

Competitive with DPO.