TRL Deep Dive

Why it matters

TRL is HF alignment library. Understanding shapes tooling.

The architecture

Trainer classes: SFTTrainer + DPOTrainer + PPOTrainer + more.

Model + tokenizer + data + config.

How it works end to end

Modern alignment: DPO + ORPO + KTO + GRPO in TRL.

Reward modeling.

Trainer patterns.