Inference-Optimal Training

Why it matters

Inference-optimal reduces cost. Popular for production LLMs.

The architecture

Smaller N + more D than Chinchilla.

Inference cost falls.

Training cost rises.

How it works end to end

Llama 3 8B on 15T tokens.

Mistral over-trained.

Inference cost dominates.