Answers are grounded in this paper and cite their sources.
Review highlights the mechanisms of generalization in deep learning, suggesting new insights into Grokking.
The phenomenon of Grokking, first reported by Power et al. [73], has gained significant attention for its unusual characteristics. Grokking is defined by a delay in generalization, during which the training loss reaches near-zero values while the test performance remains poor. This raises a critical question: if not the training loss, what drives generalization? Understanding the mechanisms behind Grokking is essential for gaining insights into the generalization process in broader deep learning contexts. In recent years, this phenomenon has attracted growing interest, with numerous theories and observations proposed to explain it. In this work, we aim to provide a comprehensive review that consolidates the current research and knowledge surrounding Grokking.
No takes yet. Share an insight, caveat, or question.
Bertolotti et al. (2026) studied this question.
Synapse has enriched 10 related papers. Consider them for comparative context: