Then and Now: Quantifying the Longitudinal Validity of Self-Disclosed Depression Diagnoses

Keith Harrigian, Mark Dredze


Abstract
Self-disclosed mental health diagnoses, which serve as ground truth annotations of mental health status in the absence of clinical measures, underpin the conclusions behind most computational studies of mental health language from the last decade. However, psychiatric conditions are dynamic; a prior depression diagnosis may no longer be indicative of an individual’s mental health, either due to treatment or other mitigating factors. We ask: to what extent are self-disclosures of mental health diagnoses actually relevant over time? We analyze recent activity from individuals who disclosed a depression diagnosis on social media over five years ago and, in turn, acquire a new understanding of how presentations of mental health status on social media manifest longitudinally. We also provide expanded evidence for the presence of personality-related biases in datasets curated using self-disclosed diagnoses. Our findings motivate three practical recommendations for improving mental health datasets curated using self-disclosed diagnoses:1) Annotate diagnosis dates and psychiatric comorbidities2) Sample control groups using propensity score matching3) Identify and remove spurious correlations introduced by selection bias
Anthology ID:
2022.clpsych-1.6
Volume:
Proceedings of the Eighth Workshop on Computational Linguistics and Clinical Psychology
Month:
July
Year:
2022
Address:
Seattle, USA
Editors:
Ayah Zirikly, Dana Atzil-Slonim, Maria Liakata, Steven Bedrick, Bart Desmet, Molly Ireland, Andrew Lee, Sean MacAvaney, Matthew Purver, Rebecca Resnik, Andrew Yates
Venue:
CLPsych
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
59–75
Language:
URL:
https://aclanthology.org/2022.clpsych-1.6
DOI:
10.18653/v1/2022.clpsych-1.6
Bibkey:
Cite (ACL):
Keith Harrigian and Mark Dredze. 2022. Then and Now: Quantifying the Longitudinal Validity of Self-Disclosed Depression Diagnoses. In Proceedings of the Eighth Workshop on Computational Linguistics and Clinical Psychology, pages 59–75, Seattle, USA. Association for Computational Linguistics.
Cite (Informal):
Then and Now: Quantifying the Longitudinal Validity of Self-Disclosed Depression Diagnoses (Harrigian & Dredze, CLPsych 2022)
Copy Citation:
PDF:
https://aclanthology.org/2022.clpsych-1.6.pdf