Expert, Crowdsourced, and Machine Assessment of Suicide Risk via Online Postings

Han-Chin Shing, Suraj Nair, Ayah Zirikly, Meir Friedenberg, Hal Daumé III, Philip Resnik


Abstract
We report on the creation of a dataset for studying assessment of suicide risk via online postings in Reddit. Evaluation of risk-level annotations by experts yields what is, to our knowledge, the first demonstration of reliability in risk assessment by clinicians based on social media postings. We also introduce and demonstrate the value of a new, detailed rubric for assessing suicide risk, compare crowdsourced with expert performance, and present baseline predictive modeling experiments using the new dataset, which will be made available to researchers through the American Association of Suicidology.
Anthology ID:
W18-0603
Volume:
Proceedings of the Fifth Workshop on Computational Linguistics and Clinical Psychology: From Keyboard to Clinic
Month:
June
Year:
2018
Address:
New Orleans, LA
Venues:
CLPsych | NAACL | WS
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
25–36
Language:
URL:
https://aclanthology.org/W18-0603
DOI:
10.18653/v1/W18-0603
Bibkey:
Cite (ACL):
Han-Chin Shing, Suraj Nair, Ayah Zirikly, Meir Friedenberg, Hal Daumé III, and Philip Resnik. 2018. Expert, Crowdsourced, and Machine Assessment of Suicide Risk via Online Postings. In Proceedings of the Fifth Workshop on Computational Linguistics and Clinical Psychology: From Keyboard to Clinic, pages 25–36, New Orleans, LA. Association for Computational Linguistics.
Cite (Informal):
Expert, Crowdsourced, and Machine Assessment of Suicide Risk via Online Postings (Shing et al., 2018)
Copy Citation:
PDF:
https://aclanthology.org/W18-0603.pdf