Which Linguist Invented the Lightbulb? Presupposition Verification for Question-Answering

Najoung Kim, Ellie Pavlick, Burcu Karagol Ayan, Deepak Ramachandran


Abstract
Many Question-Answering (QA) datasets contain unanswerable questions, but their treatment in QA systems remains primitive. Our analysis of the Natural Questions (Kwiatkowski et al. 2019) dataset reveals that a substantial portion of unanswerable questions (~21%) can be explained based on the presence of unverifiable presuppositions. Through a user preference study, we demonstrate that the oracle behavior of our proposed system—which provides responses based on presupposition failure—is preferred over the oracle behavior of existing QA systems. Then, we present a novel framework for implementing such a system in three steps: presupposition generation, presupposition verification, and explanation generation, reporting progress on each. Finally, we show that a simple modification of adding presuppositions and their verifiability to the input of a competitive end-to-end QA system yields modest gains in QA performance and unanswerability detection, demonstrating the promise of our approach.
Anthology ID:
2021.acl-long.304
Volume:
Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)
Month:
August
Year:
2021
Address:
Online
Venues:
ACL | IJCNLP
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
3932–3945
Language:
URL:
https://aclanthology.org/2021.acl-long.304
DOI:
10.18653/v1/2021.acl-long.304
Bibkey:
Copy Citation:
PDF:
https://aclanthology.org/2021.acl-long.304.pdf