Kaplan Scaling Laws (2020)

Why it matters

Kaplan started the scaling law era. Understanding evolution shapes context.

The architecture

Kaplan: GPT-3 sized for D that was too small.

Chinchilla: data starving found.

How it works end to end

Kaplan: L ~ N^-alpha with early data-fitting.

Chinchilla: revised with matched compute frontiers.

Modern: over-train for inference.