The Geometric Wall: Manifold Structure Predicts Layerwise Sparse Autoencoder Scaling Laws

arXiv:2605.09887v1 Announce Type: cross Abstract: Sparse autoencoders (SAEs) operationalise the linear representation hypothesis: they reconstruct model activations as sparse linear combinations of interpretable dictionary atoms, on the implicit assumption that activation space is well approximated...