An Exploratory Study on LLM-Generated Code and Comments in Code Repositories

The Growing Invisibility of LLM-Generated Code

A new exploratory study on arXiv (2607.01867v1) has quantified what many developers already suspect: large language models are quietly rewriting the fabric of software repositories. The paper examines LLM-generated code and comments in real-world codebases, providing empirical evidence that between 20% and 30% of code in major technology companies now originates from LLMs. This is not a future trend—it is the current baseline.

What the Research Actually Found

The study moves beyond anecdotal reports by systematically analyzing code repositories to detect LLM-generated content. While the full methodology warrants scrutiny, the core finding is significant: LLMs are no longer experimental tools but primary contributors to production code. The research specifically examines both code and accompanying comments, addressing a gap in prior work that focused narrowly on code generation alone. This dual focus matters because comments often reveal whether developers understand what the LLM produced, or are simply accepting outputs without full comprehension.

Why This Matters for Software Quality and Maintenance

The 20-30% figure carries implications that extend far beyond productivity metrics. When a quarter of a codebase is machine-generated, traditional assumptions about code ownership, debugging, and technical debt break down. LLM-generated code tends to be syntactically correct but can contain subtle logical errors, security vulnerabilities, or inefficient patterns that human reviewers may miss. More critically, LLM-generated comments often describe what code does rather than why it exists—stripping away the contextual reasoning that makes code maintainable over years.

For AI practitioners, this creates a new class of risk: the "black box codebase." When developers cannot fully explain how a system works because significant portions were generated by opaque models, debugging becomes forensic analysis rather than logical deduction. The study implicitly raises questions about whether current code review practices are adequate for LLM-generated contributions, and whether organizations are tracking which code is AI-originated.

Implications for AI Practitioners

First, teams should implement systematic tracking of LLM-generated code. Without this metadata, debugging and auditing become guesswork. Second, the study suggests that code comments—often treated as secondary—require renewed attention. If LLMs generate plausible but semantically hollow comments, the documentation layer of software degrades silently. Third, practitioners should treat LLM-generated code as a junior developer's first draft, not a final product. The 20-30% adoption rate means rigorous human review is not optional; it is the primary quality control mechanism.

Key Takeaways

• LLM-generated code now constitutes 20-30% of production code in major tech companies, making it a mainstream rather than experimental practice.
• The study's focus on both code and comments reveals a hidden risk: LLM-generated comments often lack the contextual reasoning needed for long-term maintenance.
• AI practitioners must implement metadata tracking for LLM-generated code to preserve debugging and auditing capabilities.
• Human review of LLM outputs remains essential, as syntactic correctness does not guarantee semantic or security soundness.