WriteSAE: Sparse Autoencoders for Recurrent State

arXiv:2605.12770v1 Announce Type: cross Abstract: We introduce WriteSAE, the first sparse autoencoder that decomposes and edits the matrix cache write of state-space and hybrid recurrent language models, where residual SAEs cannot reach. Existing SAEs read residual streams, but Gated DeltaNet,...