Why it matters
Grokking illuminates learning dynamics. Understanding shapes theory.
Advertisement
The architecture
Phase 1: memorization.
Long overfit training.
Phase 2: sudden generalization jump.
Advertisement
How it works end to end
Power et al. 2022.
Weight decay essential.
Adam interaction key.
Circuit-level explanation emerging.