Why it matters
Batch size shapes training. Understanding shapes design.
Advertisement
The architecture
Total tokens per step = batch * seq.
Effective via gradient accumulation.
Advertisement
How it works end to end
Per-device * world * grad-accum = effective.
Larger batch = fewer steps but same tokens.
Critical batch size: below which quality suffers.