Grokking
Why it matters
Grokking challenged the standard advice that when validation performance stalls while training performance is perfect, you should stop. In these experiments, stopping early would have missed the real solution.
It also gave researchers a clean laboratory for a deep question: when does a network memorize, and when does it learn the underlying rule? That question is at the heart of whether large models truly generalize, and grokking became one of the most studied examples in interpretability.
Deep Dive
In 2022, researchers at OpenAI reported something odd. They trained small transformers on simple math tasks such as modular addition, showing the network only part of the full table of answers. The network quickly reached 100% accuracy on the examples it had seen while scoring no better than chance on the rest: classic overfitting. But when training was left running far longer, in some cases a thousand times longer, accuracy on the unseen examples suddenly jumped to nearly 100%.
The network had stopped memorizing and started understanding. The authors called it grokking, after a word from Robert Heinlein's novel Stranger in a Strange Land meaning to understand something so thoroughly it becomes part of you. It has nothing to do with xAI's Grok model, which borrowed the same word.
What Happens Inside
In 2023, Neel Nanda and colleagues reverse-engineered a network that had grokked modular addition. It had discovered an algorithm based on trigonometry, representing numbers as rotations around a circle and combining them with trig identities. The work described three phases:
- Memorization: the network stores the training examples as a lookup table.
- Circuit formation: the general algorithm slowly forms alongside the memorized solution, with no visible change in test accuracy.
- Cleanup: the memorized solution is removed, and test accuracy suddenly jumps.
So the "sudden" understanding is gradual underneath. The curve only looks abrupt because the test metric doesn't register progress until the general circuit takes over.
Why the General Solution Wins
The leading explanation involves regularization. Memorizing needs large, intricate weights, while the general algorithm is more compact. Weight decay steadily penalizes large weights, so over time it favors the efficient solution. Without weight decay, grokking is often much slower or doesn't happen. More training data makes it happen faster, and with enough data the delay disappears.
How Far It Extends
Grokking is clearest on small algorithmic datasets. Similar delayed generalization has been produced on image and text tasks under special conditions, but large language models don't visibly behave this way, since they rarely see the same example twice and so can't memorize first. Related effects include double descent, where test error rises and then falls again as models or training time grow.
Lessons for Practitioners
- Don't keep training real-world models forever hoping they'll grok. In ordinary fine-tuning on small datasets, a widening gap between training and validation loss is still just overfitting.
- A flat metric can hide progress. Loss curves and accuracy numbers don't show everything happening in the weights.
- Regularization doesn't only prevent overfitting; it can change which solution a network finds.