Benchmarking for Public Health Surveillance tasks on Social Media with a Domain-Specific Pretrained Language Model

Usman Naseem, Byoung Chan Lee, Matloob Khushi, Jinman Kim, Adam Dunn


Abstract
A user-generated text on social media enables health workers to keep track of information, identify possible outbreaks, forecast disease trends, monitor emergency cases, and ascertain disease awareness and response to official health correspondence. This exchange of health information on social media has been regarded as an attempt to enhance public health surveillance (PHS). Despite its potential, the technology is still in its early stages and is not ready for widespread application. Advancements in pretrained language models (PLMs) have facilitated the development of several domain-specific PLMs and a variety of downstream applications. However, there are no PLMs for social media tasks involving PHS. We present and release PHS-BERT, a transformer-based PLM, to identify tasks related to public health surveillance on social media. We compared and benchmarked the performance of PHS-BERT on 25 datasets from different social medial platforms related to 7 different PHS tasks. Compared with existing PLMs that are mainly evaluated on limited tasks, PHS-BERT achieved state-of-the-art performance on all 25 tested datasets, showing that our PLM is robust and generalizable in the common PHS tasks. By making PHS-BERT available, we aim to facilitate the community to reduce the computational cost and introduce new baselines for future works across various PHS-related tasks.
Anthology ID:
2022.nlppower-1.3
Volume:
Proceedings of NLP Power! The First Workshop on Efficient Benchmarking in NLP
Month:
May
Year:
2022
Address:
Dublin, Ireland
Venues:
ACL | nlppower
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
22–31
Language:
URL:
https://aclanthology.org/2022.nlppower-1.3
DOI:
10.18653/v1/2022.nlppower-1.3
Bibkey:
Cite (ACL):
Usman Naseem, Byoung Chan Lee, Matloob Khushi, Jinman Kim, and Adam Dunn. 2022. Benchmarking for Public Health Surveillance tasks on Social Media with a Domain-Specific Pretrained Language Model. In Proceedings of NLP Power! The First Workshop on Efficient Benchmarking in NLP, pages 22–31, Dublin, Ireland. Association for Computational Linguistics.
Cite (Informal):
Benchmarking for Public Health Surveillance tasks on Social Media with a Domain-Specific Pretrained Language Model (Naseem et al., nlppower 2022)
Copy Citation:
PDF:
https://aclanthology.org/2022.nlppower-1.3.pdf
Data
DreadditPUBHEALTH