Training Data Mixing

Why it matters

Data mixing shapes model capabilities. Proxy studies guide big runs.

The architecture

Choose domains + weights.

Proxy studies at small scale.

Apply to big.

How it works end to end

DoReMi.

DoGE, RegMix.

Small proxy studies.