Can Large Language Models Identify Implicit Suicidal Ideation? An Empirical Evaluation

Suicide remains a major global mental health challenge, and early intervention hinges on recognizing signs of suicidal ideation. In private conversations, such ideation is often expressed in subtle or conflicted ways, making detection especially difficult. Existing data sets are mainly based on public help-seeking platforms such as Reddit, which fail to capture the introspective and ambiguous nature of suicidal ideation in more private contexts. To address this gap, we introduce , a novel dataset of 1,200 test cases simulating implicit suicidal ideation within psychologically rich dialogue scenarios. Each case is grounded in psychological theory, combining the Death/Suicide Implicit Association Test (D/S-IAT) patterns, expanded suicidal expressions, cognitive distortions, and contextual stressors. In addition, we propose a psychology-guided evaluation framework to assess the ability of LLMs to identify implicit suicidal ideation through their responses. Experiments with eight widely used LLMs across varied prompting conditions reveal that current models often struggle significantly to recognize implicit suicidal ideation. Our findings highlight the urgent need for more clinically grounded evaluation frameworks and design practices to ensure the safe use of LLMs in sensitive support systems.

Citation

T. Li, S. Yang, J. Wu, J. Wei, L. Hu, M. Li, D.F. Wong, J.R. Oltmanns, D. Wang, "Can Large Language Models Identify Implicit Suicidal Ideation? An Empirical Evaluation," 2025, pp. 18392-18413.

Source

EMNLP 2025 - 2025 Conference on Empirical Methods in Natural Language Processing, Findings of EMNLP 2025

Conference

Findings of the Association for Computational Linguistics: EMNLP 2025

Source

Findings of the Association for Computational Linguistics: EMNLP 2025

Publisher

Association for Computational Linguistics (ACL)

DOI

10.18653/v1/2025.findings-emnlp.998

Additional links

https://aclanthology.org/2025.findings-emnlp.998/