The Geometric Inductive Bias of Grokking: Bypassing Phase Transitions via Architectural Topology

arXiv:2603.05228v3 Announce Type: replace-cross Abstract: Mechanistic interpretability typically relies on post-hoc analysis of trained networks. We instead adopt an interventional approach: testing hypotheses a priori by modifying architectural topology to observe training dynamics. We study...