Responses of AI chatbots to escalating suicide risk: A simulation study of repeated interactions
文献摘要
Background Generative AI chatbots are increasingly used for emotional support, yet their safety remains insufficiently characterized across repeated interactions. Prior work relied mostly on single-session prompts, limiting evaluation of changes in responses across days. The present study aimed to assess how major AI chatbots respond to escalating suicidal risk over time, including imminent-risk scenarios and attempts to bypass safeguards (jailbreaking). Methods We simulated interactions with ChatGPT, DeepSeek, and Replika across three suicidal risk scenarios with risk escalation over seven days, informed by the Columbia Suicide Severity Rating Scale. Outcomes included exploration of suicidal ideation, symptom assessment, referral to human support, and vulnerability to jailbreak attempts. Results In ChatGPT, DeepSeek, and Replika, respectively, overall human referral occurred in 54/63 (85.7%), 48/63 (76.2%), and 6/63 (9.5%) daily records. During the high-risk period, exploration of suicidal ideation was higher for ChatGPT than DeepSeek (adjusted difference, 25.9 percentage points; 95% CI, 2.4-49.5; q = 0.037) and Replika (66.7 percentage points; 95% CI, 44.2-89.1; q < 0.001). Day 7 referral occurred in 9/9 (exact 95% CI 66.4-100%), 4/9 (13.7-78.8%), and 1/9 (0.3-48.2%) trajectories, respectively. Jailbreaking succeeded in 6/9 (exact 95% CI 29.9-92.5%), 7/9 (40.0-97.2%), and 8/9 (51.8-99.7%) attempts, respectively. Conclusions AI chatbots showed marked variability and safety vulnerabilities in simulated suicidal crises, particularly under jailbreaking conditions. The simulation design and small sample of 27 trajectories limit generalizability. These findings raise concerns about unsupervised chatbot use during suicidal crises and support further evaluation of safeguards across repeated interactions.