Simple and Effective Semi-Supervised Question Answering

Bhuwan Dhingra, Danish Pruthi, Dheeraj Rajagopal


Abstract
Recent success of deep learning models for the task of extractive Question Answering (QA) is hinged on the availability of large annotated corpora. However, large domain specific annotated corpora are limited and expensive to construct. In this work, we envision a system where the end user specifies a set of base documents and only a few labelled examples. Our system exploits the document structure to create cloze-style questions from these base documents; pre-trains a powerful neural network on the cloze style questions; and further fine-tunes the model on the labeled examples. We evaluate our proposed system across three diverse datasets from different domains, and find it to be highly effective with very little labeled data. We attain more than 50% F1 score on SQuAD and TriviaQA with less than a thousand labelled examples. We are also releasing a set of 3.2M cloze-style questions for practitioners to use while building QA systems.
Anthology ID:
N18-2092
Volume:
Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers)
Month:
June
Year:
2018
Address:
New Orleans, Louisiana
Editors:
Marilyn Walker, Heng Ji, Amanda Stent
Venue:
NAACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
582–587
Language:
URL:
https://aclanthology.org/N18-2092
DOI:
10.18653/v1/N18-2092
Bibkey:
Cite (ACL):
Bhuwan Dhingra, Danish Pruthi, and Dheeraj Rajagopal. 2018. Simple and Effective Semi-Supervised Question Answering. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers), pages 582–587, New Orleans, Louisiana. Association for Computational Linguistics.
Cite (Informal):
Simple and Effective Semi-Supervised Question Answering (Dhingra et al., NAACL 2018)
Copy Citation:
PDF:
https://aclanthology.org/N18-2092.pdf
Data
BioASQSQuADTriviaQA