Why it matters

HP transfer changes economics of LLM. Understanding shapes strategy.

Advertisement

The architecture

muP: makes HPs invariant.

Scaling laws: extrapolate quality.

Small sweeps + apply to large.

HP transfer stackSmall proxysweep HPsmuP + lawsinvariant + extrapolateLarge runpredicted HPsCerebras + Anthropic + Microsoft use variants; frontier standard
HP transfer.
Advertisement

How it works end to end

Sweep proxy: 100M-500M model.

Apply learned HPs to 70B+.

Verify on scaling proxies.