Why it matters

ORPO simplifies alignment pipeline. Understanding shapes design.

Advertisement

The architecture

SFT loss + odds-ratio term.

No separate reference model.

Alpha tunes weights.

ORPO FT flowSFT losson chosenOdds ratiochosen vs rejectedCombinedsingle objectiveHong et al. 2024; one-stage vs SFT + DPO; no reference model
ORPO FT.
Advertisement

How it works end to end

Alpha 0.1 typical.

PEFT compatible.

Faster than SFT + DPO.