One Year Later...The Harms Persist, But So Do We!

The Unfinished Safety Agenda in Mental Health AI

A new preprint on arXiv (2606.23884v1) presents a sobering one-year follow-up evaluation of six proprietary large language models used in mental health contexts. The study systematically tests these models across 16 clinical conditions, finding that safety safeguards remain “inadequate and inconsistent” — a finding that echoes earlier concerns but now carries the weight of a longitudinal benchmark.

What the Study Reveals

The research team assessed how leading LLMs handle mental health conversations, ranging from depression and anxiety to more acute conditions like suicidal ideation or psychosis. The core finding is not that models are universally dangerous, but that their safety behaviors vary unpredictably by condition and model version. Some models may offer reasonable responses for common mood disorders while failing catastrophically on crisis-related queries. This inconsistency is arguably more troubling than uniform failure, because it creates a false sense of reliability.

The “one year later” framing is critical. It suggests that despite public pressure, regulatory attention, and voluntary commitments from AI labs, the fundamental safety gap in mental health applications has not closed. Updates to model weights and system prompts have not produced systematic improvements across the clinical spectrum.

Why This Matters

Mental health is a domain where the stakes are uniquely high. An LLM that gives poor coding advice can be corrected; one that mishandles a user expressing suicidal thoughts can have irreversible consequences. The study’s findings undermine the notion that “safety” is a solved problem for general-purpose models. Instead, safety appears to be a patchwork — effective in some areas, porous in others, and often opaque to end users.

For regulators, this raises a difficult question: should general-purpose LLMs be allowed to engage in mental health conversations at all without condition-specific certification? The inconsistency documented here suggests that current self-regulation is insufficient.

Implications for AI Practitioners

For developers deploying LLMs in health-adjacent contexts, the takeaway is clear: do not assume safety generalizes. A model that passes broad red-teaming may still fail on niche clinical presentations. Practitioners should:

• Implement condition-specific guardrails: A single safety classifier or system prompt is unlikely to cover the full range of mental health risks. Separate handling for crisis, chronic conditions, and subclinical support is necessary.
• Conduct longitudinal testing: Model updates can silently degrade safety performance. Regular re-evaluation across the full clinical spectrum is essential, not just on a handful of benchmark prompts.
• Design for escalation: No LLM should be the final word in mental health. Systems must have clear, low-friction pathways to human professionals, especially when risk indicators are detected.

Key Takeaways

• Six proprietary LLMs still show inconsistent safety performance across 16 mental health conditions one year after initial concerns were raised.
• Safety failures are not uniform — models may perform well on common conditions but poorly on crisis scenarios, creating a misleading sense of reliability.
• AI practitioners must implement condition-specific guardrails and conduct longitudinal safety testing, rather than relying on general red-teaming.
• The findings argue against deploying general-purpose LLMs in mental health without domain-specific certification and clear escalation protocols.