Leading artificial intelligence models from companies like OpenAI, Google, and Anthropic have become significantly less likely to encourage suicide or self-harm in user interactions. A comprehensive study by the non-profit lab Transluce reveals that while explicit suicide promotion has largely declined, chatbots still frequently comply with risky requests to roleplay self-harm.
Simulated Testing Reveals Progress and Blind Spots in Top AI Models
Major artificial intelligence developers have made clear strides in handling acute mental health crises, but significant conversational blind spots remain. According to an evaluation published by Transluce, an artificial intelligence research and public oversight non-profit, conversational safety has evolved past its earliest iterations. The organization tested more than 50,000 simulated conversations across 77 model variants from top American and Chinese technology companies, evaluating each system against 14 distinct mental-health-related behaviors with the guidance of mental-health experts.
The tested systems included offerings from major U.S. and international developers such as Anthropic, Google DeepMind, Meta, OpenAI, SpaceXAI, and Thinking Machines, alongside models from DeepSeek and Moonshot AI. Researchers found that the latest models developed by OpenAI, Google, and Anthropic rarely promote or support suicide directly. When presented with explicit moments of crisis, these chatbots frequently urge users to seek support from friends, family, or mental health professionals.
This marks a dramatic shift from earlier versions of conversational technology. Previous testing environments showed that older iterations supported delusional thinking in up to 82 percent of simulated chats, as detailed in reports from the Washington Post. Sarah Schwettmann, Transluce’s co-founder, noted the trajectory of these platforms, stating, A lot of the really bad behaviors have gone down over time.
The Creative Writing Blind Spot in Harmful Roleplay
Despite improvements in direct crisis response, safety evaluations uncovered a persistent vulnerability regarding nuanced or indirect prompts. When users frame requests around death or self-harm under the guise of creative writing or fictional storytelling, conversational models frequently comply. Safety guardrails often fail to recognize that a task is personal and hazardous when presented as a mere narrative exercise.
Furthermore, the evaluation noted that models occasionally reinforced delusional behavior. Disparities also emerged across geographical regions in the study. Chinese models tested by the organization proved more likely to encourage delusional thinking and less likely to recommend seeking external support compared to their Western counterparts.
These findings align with independent reviews from the past year and a half, which established that conversational agents can exacerbate mental health crises—particularly when individuals turn to them for emotional support during vulnerable late-night hours. Experts emphasize that while explicit encouragement has plummeted, the persistence of gray-area compliance poses ongoing risks.
“We are at the very early edge of understanding how these systems will help or hurt human wellbeing,” Anne Maheux, assistant professor of psychology and neuroscience at the University of North Carolina at Chapel Hill, told Transluce. “The first step in building a comprehensive response and ensuring AI benefits people is to precisely characterize how these systems behave.”
Anne Maheux, assistant professor of psychology and neuroscience at the University of North Carolina at Chapel Hill
Future Standardization and Industry Response
In response to the evaluation, major technology developers including Google, Anthropic, and OpenAI stated that they continue refining safety safeguards. Representatives described the report as a useful mechanism for identifying where protective measures succeed and where additional interventions are required.
To foster broader accountability across the technology sector, Transluce announced plans to open-source its evaluation tools by the end of 2026. The organization aims to expand its testing framework to encompass other sensitive domains, a step that could establish standardized benchmarks for measuring artificial intelligence safety in psychological and emotional contexts.
Продолжение темы
