The two clocks and the innovation window: When and how generative models learn rules

arXiv:2605.10019v1 Announce Type: cross Abstract: Generative models trained on finite data face a fundamental tension: their score-matching or next-token objective converges to the empirical training distribution rather than the population distribution we seek to learn. Using rule-valid synthetic...