Conversation-Aware Filtering of Online Patient Forum Messages

Anne Dirkson, Suzan Verberne, Wessel Kraaij


Abstract
Previous approaches to NLP tasks on online patient forums have been limited to single posts as units, thereby neglecting the overarching conversational structure. In this paper we explore the benefit of exploiting conversational context for filtering posts relevant to a specific medical topic. We experiment with two approaches to add conversational context to a BERT model: a sequential CRF layer and manually engineered features. Although neither approach can outperform the F1 score of the BERT baseline, we find that adding a sequential layer improves precision for all target classes whereas adding a non-sequential layer with manually engineered features leads to a higher recall for two out of three target classes. Thus, depending on the end goal, conversation-aware modelling may be beneficial for identifying relevant messages. We hope our findings encourage other researchers in this domain to move beyond studying messages in isolation towards more discourse-based data collection and classification. We release our code for the purpose of follow-up research.
Anthology ID:
2020.smm4h-1.2
Volume:
Proceedings of the Fifth Social Media Mining for Health Applications Workshop & Shared Task
Month:
December
Year:
2020
Address:
Barcelona, Spain (Online)
Editors:
Graciela Gonzalez-Hernandez, Ari Z. Klein, Ivan Flores, Davy Weissenbacher, Arjun Magge, Karen O'Connor, Abeed Sarker, Anne-Lyse Minard, Elena Tutubalina, Zulfat Miftahutdinov, Ilseyar Alimova
Venue:
SMM4H
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
11–18
Language:
URL:
https://aclanthology.org/2020.smm4h-1.2
DOI:
Bibkey:
Cite (ACL):
Anne Dirkson, Suzan Verberne, and Wessel Kraaij. 2020. Conversation-Aware Filtering of Online Patient Forum Messages. In Proceedings of the Fifth Social Media Mining for Health Applications Workshop & Shared Task, pages 11–18, Barcelona, Spain (Online). Association for Computational Linguistics.
Cite (Informal):
Conversation-Aware Filtering of Online Patient Forum Messages (Dirkson et al., SMM4H 2020)
Copy Citation:
PDF:
https://aclanthology.org/2020.smm4h-1.2.pdf