A Hierarchical Language Model with Predictable Scaling Laws and Provable Benefits of Reasoning

arXiv:2605.13687v1 Announce Type: cross Abstract: We introduce a family of synthetic languages with hierarchical structure -- generated by a broadcast process on trees -- for which the role of context length and reasoning in autoregressive generation can be analyzed precisely. At the heart of our...