A Geometric Perspective on Next-Token Prediction in Large Language Models: Three Emerging Phases

arXiv:2605.09011v1 Announce Type: cross Abstract: We investigate the geometry of predictive information across the layers of large language models (LLMs). We repurpose representation lenses-learned affine maps trained to predict the next token from intermediate residual streams-as geometric...