Why it matters

Transitions shape training economics. Understanding shapes design.

Advertisement

The architecture

Capabilities emerge at different points.

Some monotone; some U-shaped.

Data mix + curriculum affect.

Capability transitionsEarlysurface statsMidsyntax + factsLatereasoning + skillsRough phenomenological description; internal circuits story more complex
Capability transitions.
Advertisement

How it works end to end

Early: bigrams, syntax.

Mid: factual recall, translation.

Late: reasoning, tool use.

Curriculum affects order.