Why it matters

Alignment tax trades safety for capability. Careful design minimizes.

Advertisement

The architecture

Measure capability before/after.

Diff = tax.

KL + SFT mixing helps.

Alignment tax mathMeasure baselinecapabilityMeasure alignedpost-trainingDiff = taxwatch closelyInstructGPT paper measured tax; KL + SFT mixing minimize
Alignment tax math.
Advertisement

How it works end to end

Measure benchmarks.

KL + SFT mixing.

Targeted alignment.