SB_NITK at MEDIQA 2021: Leveraging Transfer Learning for Question Summarization in Medical Domain

Spandana Balumuri, Sony Bachina, Sowmya Kamath S


Abstract
Recent strides in the healthcare domain, have resulted in vast quantities of streaming data available for use for building intelligent knowledge-based applications. However, the challenges introduced to the huge volume, velocity of generation, variety and variability of this medical data have to be adequately addressed. In this paper, we describe the model and results for our submission at MEDIQA 2021 Question Summarization shared task. In order to improve the performance of summarization of consumer health questions, our method explores the use of transfer learning to utilize the knowledge of NLP transformers like BART, T5 and PEGASUS. The proposed models utilize the knowledge of pre-trained NLP transformers to achieve improved results when compared to conventional deep learning models such as LSTM, RNN etc. Our team SB_NITK ranked 12th among the total 22 submissions in the official final rankings. Our BART based model achieved a ROUGE-2 F1 score of 0.139.
Anthology ID:
2021.bionlp-1.31
Volume:
Proceedings of the 20th Workshop on Biomedical Language Processing
Month:
June
Year:
2021
Address:
Online
Editors:
Dina Demner-Fushman, Kevin Bretonnel Cohen, Sophia Ananiadou, Junichi Tsujii
Venue:
BioNLP
SIG:
SIGBIOMED
Publisher:
Association for Computational Linguistics
Note:
Pages:
273–279
Language:
URL:
https://aclanthology.org/2021.bionlp-1.31
DOI:
10.18653/v1/2021.bionlp-1.31
Bibkey:
Cite (ACL):
Spandana Balumuri, Sony Bachina, and Sowmya Kamath S. 2021. SB_NITK at MEDIQA 2021: Leveraging Transfer Learning for Question Summarization in Medical Domain. In Proceedings of the 20th Workshop on Biomedical Language Processing, pages 273–279, Online. Association for Computational Linguistics.
Cite (Informal):
SB_NITK at MEDIQA 2021: Leveraging Transfer Learning for Question Summarization in Medical Domain (Balumuri et al., BioNLP 2021)
Copy Citation:
PDF:
https://aclanthology.org/2021.bionlp-1.31.pdf
Data
C4MeQSum