知镜 Cognispect · 医学文献与研究工作台

Responses of AI chatbots to escalating suicide risk: A simulation study of repeated interactions

Pichowicz W, Kotas M, Kalka N, et al.
J Affect Disord · 2026-10-05 · PMID 42833407

文献摘要

Background Generative AI chatbots are increasingly used for emotional support, yet their safety remains insufficiently characterized across repeated interactions. Prior work relied mostly on single-session prompts, limiting evaluation of changes in responses across days. The present study aimed to assess how major AI chatbots respond to escalating suicidal risk over time, including imminent-risk scenarios and attempts to bypass safeguards (jailbreaking). Methods We simulated interactions with ChatGPT, DeepSeek, and Replika across three suicidal risk scenarios with risk escalation over seven days, informed by the Columbia Suicide Severity Rating Scale. Outcomes included exploration of suicidal ideation, symptom assessment, referral to human support, and vulnerability to jailbreak attempts. Results In ChatGPT, DeepSeek, and Replika, respectively, overall human referral occurred in 54/63 (85.7%), 48/63 (76.2%), and 6/63 (9.5%) daily records. During the high-risk period, exploration of suicidal ideation was higher for ChatGPT than DeepSeek (adjusted difference, 25.9 percentage points; 95% CI, 2.4-49.5; q = 0.037) and Replika (66.7 percentage points; 95% CI, 44.2-89.1; q < 0.001). Day 7 referral occurred in 9/9 (exact 95% CI 66.4-100%), 4/9 (13.7-78.8%), and 1/9 (0.3-48.2%) trajectories, respectively. Jailbreaking succeeded in 6/9 (exact 95% CI 29.9-92.5%), 7/9 (40.0-97.2%), and 8/9 (51.8-99.7%) attempts, respectively. Conclusions AI chatbots showed marked variability and safety vulnerabilities in simulated suicidal crises, particularly under jailbreaking conditions. The simulation design and small sample of 27 trajectories limit generalizability. These findings raise concerns about unsupervised chatbot use during suicidal crises and support further evaluation of safeguards across repeated interactions.

在知镜精读原文来源DOI 原文

知镜提供医学文献检索、阅读笔记、人体图谱、统计绘图与研究复现工具。