Transformer Models for Drug Adverse Effects Detection from Tweets

Pavel Blinov, Manvel Avetisian


Abstract
In this paper we present the drug adverse effects detection system developed during our participation in the Social Media Mining for Health Applications Shared Task 2020. We experimented with transfer learning approach for English and Russian, BERT and RoBERTa architectures and several strategies for regression head composition. Our final submissions in both languages overcome average F1 by several percents margin.
Anthology ID:
2020.smm4h-1.17
Volume:
Proceedings of the Fifth Social Media Mining for Health Applications Workshop & Shared Task
Month:
December
Year:
2020
Address:
Barcelona, Spain (Online)
Editors:
Graciela Gonzalez-Hernandez, Ari Z. Klein, Ivan Flores, Davy Weissenbacher, Arjun Magge, Karen O'Connor, Abeed Sarker, Anne-Lyse Minard, Elena Tutubalina, Zulfat Miftahutdinov, Ilseyar Alimova
Venue:
SMM4H
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
110–112
Language:
URL:
https://aclanthology.org/2020.smm4h-1.17
DOI:
Bibkey:
Cite (ACL):
Pavel Blinov and Manvel Avetisian. 2020. Transformer Models for Drug Adverse Effects Detection from Tweets. In Proceedings of the Fifth Social Media Mining for Health Applications Workshop & Shared Task, pages 110–112, Barcelona, Spain (Online). Association for Computational Linguistics.
Cite (Informal):
Transformer Models for Drug Adverse Effects Detection from Tweets (Blinov & Avetisian, SMM4H 2020)
Copy Citation:
PDF:
https://aclanthology.org/2020.smm4h-1.17.pdf