Reducing Inference Time of Biomedical NER Tasks using Multi-Task Learning

Mukund Chaudhry Chaudhry, Arman Kazmi, Shashank Jatav, Akhilesh Verma, Vishal Samal, Kristopher Paul, Ashutosh Modi


Abstract
Recently, fine-tuned transformer-based models (e.g., PubMedBERT, BioBERT) have shown the state-of-the-art performance of a number of BioNLP tasks, such as Named Entity Recognition (NER). However, transformer-based models are complex and have millions of parameters, and, consequently, are relatively slow during inference. In this paper, we address the time complexity limitations of the BioNLP transformer models. In particular, we propose a Multi-Task Learning based framework for jointly learning three different biomedical NER tasks. Our experiments show a reduction in inference time by a factor of three without any reduction in prediction accuracy.
Anthology ID:
2022.icon-main.16
Volume:
Proceedings of the 19th International Conference on Natural Language Processing (ICON)
Month:
December
Year:
2022
Address:
New Delhi, India
Editors:
Md. Shad Akhtar, Tanmoy Chakraborty
Venue:
ICON
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
116–122
Language:
URL:
https://aclanthology.org/2022.icon-main.16
DOI:
Bibkey:
Cite (ACL):
Mukund Chaudhry Chaudhry, Arman Kazmi, Shashank Jatav, Akhilesh Verma, Vishal Samal, Kristopher Paul, and Ashutosh Modi. 2022. Reducing Inference Time of Biomedical NER Tasks using Multi-Task Learning. In Proceedings of the 19th International Conference on Natural Language Processing (ICON), pages 116–122, New Delhi, India. Association for Computational Linguistics.
Cite (Informal):
Reducing Inference Time of Biomedical NER Tasks using Multi-Task Learning (Chaudhry et al., ICON 2022)
Copy Citation:
PDF:
https://aclanthology.org/2022.icon-main.16.pdf