ALBA: Adaptive Language-Based Assessments for Mental Health

Vasudha Varadarajan, Sverker Sikström, Oscar Kjell, H. Schwartz


Abstract
Mental health issues differ widely among individuals, with varied signs and symptoms. Recently, language-based assessments haveshown promise in capturing this diversity, but they require a substantial sample of words per person for accuracy. This work introducesthe task of Adaptive Language-Based Assessment (ALBA), which involves adaptively ordering questions while also scoring an individual’s latent psychological trait using limited language responses to previous questions. To this end, we develop adaptive testing methods under two psychometric measurement theories: Classical Test Theory and Item Response Theory.We empirically evaluate ordering and scoring strategies, organizing into two new methods: a semi-supervised item response theory-basedmethod (ALIRT) and a supervised Actor-Critic model. While we found both methods to improve over non-adaptive baselines, We foundALIRT to be the most accurate and scalable, achieving the highest accuracy with fewer questions (e.g., Pearson r ≈ 0.93 after only 3 questions as compared to typically needing at least 7 questions). In general, adaptive language-based assessments of depression and anxiety were able to utilize a smaller sample of language without compromising validity or large computational costs.
Anthology ID:
2024.naacl-long.136
Volume:
Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers)
Month:
June
Year:
2024
Address:
Mexico City, Mexico
Editors:
Kevin Duh, Helena Gomez, Steven Bethard
Venue:
NAACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
2466–2478
Language:
URL:
https://aclanthology.org/2024.naacl-long.136
DOI:
Bibkey:
Cite (ACL):
Vasudha Varadarajan, Sverker Sikström, Oscar Kjell, and H. Schwartz. 2024. ALBA: Adaptive Language-Based Assessments for Mental Health. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), pages 2466–2478, Mexico City, Mexico. Association for Computational Linguistics.
Cite (Informal):
ALBA: Adaptive Language-Based Assessments for Mental Health (Varadarajan et al., NAACL 2024)
Copy Citation:
PDF:
https://aclanthology.org/2024.naacl-long.136.pdf
Copyright:
 2024.naacl-long.136.copyright.pdf