Using Natural Language Relations between Answer Choices for Machine Comprehension

Rajkumar Pujari, Dan Goldwasser


Abstract
While evaluating an answer choice for Reading Comprehension task, other answer choices available for the question and the answers of related questions about the same paragraph often provide valuable information. In this paper, we propose a method to leverage the natural language relations between the answer choices, such as entailment and contradiction, to improve the performance of machine comprehension. We use a stand-alone question answering (QA) system to perform QA task and a Natural Language Inference (NLI) system to identify the relations between the choice pairs. Then we perform inference using an Integer Linear Programming (ILP)-based relational framework to re-evaluate the decisions made by the standalone QA system in light of the relations identified by the NLI system. We also propose a multitask learning model that learns both the tasks jointly.
Anthology ID:
N19-1403
Volume:
Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)
Month:
June
Year:
2019
Address:
Minneapolis, Minnesota
Editors:
Jill Burstein, Christy Doran, Thamar Solorio
Venue:
NAACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
4010–4015
Language:
URL:
https://aclanthology.org/N19-1403
DOI:
10.18653/v1/N19-1403
Bibkey:
Cite (ACL):
Rajkumar Pujari and Dan Goldwasser. 2019. Using Natural Language Relations between Answer Choices for Machine Comprehension. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 4010–4015, Minneapolis, Minnesota. Association for Computational Linguistics.
Cite (Informal):
Using Natural Language Relations between Answer Choices for Machine Comprehension (Pujari & Goldwasser, NAACL 2019)
Copy Citation:
PDF:
https://aclanthology.org/N19-1403.pdf
Presentation:
 N19-1403.Presentation.pdf
Code
 pujari-rajkumar/naacl2019
Data
MultiRCSNLI