When AI Reviews Its Own Code: Recursive Self-Training Collapse in Code LLMs

When AI Trains on Its Own Output: The Hidden Risk of Code Model Collapse

A new preprint from arXiv (2606.28438) has systematically demonstrated what many developers have feared: recursive self-training—where AI models are trained on code they previously generated—leads to measurable degradation in output quality. The researchers studied code-focused large language models (LLMs) and found that without fresh human-generated data or external quality filters, performance declines in a pattern they term “recursive self-training collapse.”

The mechanism is straightforward but insidious. When AI-generated code enters public repositories (e.g., through automated pull requests or developer adoption), it later gets scraped and included in training datasets for the next generation of models. This creates a feedback loop: models learn from their own increasingly flawed outputs, amplifying errors, edge-case failures, and stylistic quirks rather than correcting them. The study shows that even a few cycles of this recursion can reduce functional correctness, especially for complex or less-common programming tasks.

Why This Matters Beyond the Lab

This finding has immediate practical consequences. Code LLMs are already integrated into CI/CD pipelines, code review tools, and even autonomous coding agents. If those systems generate code that later contaminates training data, the entire ecosystem could experience a slow, silent decline in reliability. The problem is compounded by the fact that most quality benchmarks test on curated, static datasets—they do not capture the dynamic degradation that occurs when models are repeatedly exposed to their own outputs.

For organizations building or fine-tuning code models, the implication is clear: you cannot treat all code as equally valuable for training. Code from repositories that heavily use AI assistance may carry latent quality issues that only compound over time. The paper suggests that without explicit provenance tracking or quality gates, the very efficiency gains from AI-generated code could undermine the long-term health of the models themselves.

Implications for AI Practitioners

First, provenance matters. Teams training code models should filter training data based on whether it was human-written or AI-generated, or at least apply quality checks before inclusion. Second, self-training must be bounded. If you fine-tune on model outputs, you need a control mechanism—either periodic human validation or external test suites that catch regressions. Third, monitor for distribution shift. A model that seems stable on standard benchmarks may still be degrading on long-tail tasks, which is exactly where recursive collapse first appears.

The study does not argue that self-training is always harmful—only that it is risky without safeguards. For practitioners, the takeaway is not to abandon AI-assisted coding but to treat generated code as a resource that requires curation, not as a free lunch for future training.

Key Takeaways

• Recursive self-training on AI-generated code leads to measurable performance degradation, especially on complex or rare tasks.
• Code from AI-assisted repositories can contaminate future training data, creating a feedback loop of declining quality.
• Practitioners should implement provenance tracking and quality gates when using generated code for training.
• Standard benchmarks may mask this degradation; monitoring long-tail performance is essential for detecting collapse early.