Overview of the MEDIQA 2019 Shared Task on Textual Inference, Question Entailment and Question Answering

Asma Ben Abacha, Chaitanya Shivade, Dina Demner-Fushman


Abstract
This paper presents the MEDIQA 2019 shared task organized at the ACL-BioNLP workshop. The shared task is motivated by a need to develop relevant methods, techniques and gold standards for inference and entailment in the medical domain, and their application to improve domain specific information retrieval and question answering systems. MEDIQA 2019 includes three tasks: Natural Language Inference (NLI), Recognizing Question Entailment (RQE), and Question Answering (QA) in the medical domain. 72 teams participated in the challenge, achieving an accuracy of 98% in the NLI task, 74.9% in the RQE task, and 78.3% in the QA task. In this paper, we describe the tasks, the datasets, and the participants’ approaches and results. We hope that this shared task will attract further research efforts in textual inference, question entailment, and question answering in the medical domain.
Anthology ID:
W19-5039
Volume:
Proceedings of the 18th BioNLP Workshop and Shared Task
Month:
August
Year:
2019
Address:
Florence, Italy
Venues:
ACL | BioNLP | WS
SIG:
SIGBIOMED
Publisher:
Association for Computational Linguistics
Note:
Pages:
370–379
Language:
URL:
https://aclanthology.org/W19-5039
DOI:
10.18653/v1/W19-5039
Bibkey:
Cite (ACL):
Asma Ben Abacha, Chaitanya Shivade, and Dina Demner-Fushman. 2019. Overview of the MEDIQA 2019 Shared Task on Textual Inference, Question Entailment and Question Answering. In Proceedings of the 18th BioNLP Workshop and Shared Task, pages 370–379, Florence, Italy. Association for Computational Linguistics.
Cite (Informal):
Overview of the MEDIQA 2019 Shared Task on Textual Inference, Question Entailment and Question Answering (Ben Abacha et al., 2019)
Copy Citation:
PDF:
https://aclanthology.org/W19-5039.pdf
Code
 abachaa/MEDIQA2019
Data
GLUEMedQuADMultiNLI