Spectral Entropy Collapse as an Empirical Signature of Delayed Generalisation in Grokking

arXiv:2604.13123v1 Announce Type: cross Abstract: Grokking -- delayed generalisation long after memorisation -- lacks a predictive mechanistic explanation. We identify the normalised spectral entropy $\tilde{H}(t)$ of the representation covariance as a scalar order parameter for this transition,...