Why it matters

Batch size shapes training. Understanding shapes design.

Advertisement

The architecture

Total tokens per step = batch * seq.

Effective via gradient accumulation.

Batch size hierarchyPer-GPU batchseq * micro-batchGlobal batchworld size * ...Grad accumextend effectiveFrontier LLM: 4M+ tokens per step; Chinchilla ~2M; GPT-3 3.2M
Batch size.
Advertisement

How it works end to end

Per-device * world * grad-accum = effective.

Larger batch = fewer steps but same tokens.

Critical batch size: below which quality suffers.