Do Multi-Hop Question Answering Systems Know How to Answer the Single-Hop Sub-Questions?

Yixuan Tang, Hwee Tou Ng, Anthony Tung


Abstract
Multi-hop question answering (QA) requires a model to retrieve and integrate information from multiple passages to answer a question. Rapid progress has been made on multi-hop QA systems with regard to standard evaluation metrics, including EM and F1. However, by simply evaluating the correctness of the answers, it is unclear to what extent these systems have learned the ability to perform multi-hop reasoning. In this paper, we propose an additional sub-question evaluation for the multi-hop QA dataset HotpotQA, in order to shed some light on explaining the reasoning process of QA systems in answering complex questions. We adopt a neural decomposition model to generate sub-questions for a multi-hop question, followed by extracting the corresponding sub-answers. Contrary to our expectation, multiple state-of-the-art multi-hop QA models fail to answer a large portion of sub-questions, although the corresponding multi-hop questions are correctly answered. Our work takes a step forward towards building a more explainable multi-hop QA system.
Anthology ID:
2021.eacl-main.283
Volume:
Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume
Month:
April
Year:
2021
Address:
Online
Venue:
EACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
3244–3249
Language:
URL:
https://aclanthology.org/2021.eacl-main.283
DOI:
10.18653/v1/2021.eacl-main.283
Bibkey:
Cite (ACL):
Yixuan Tang, Hwee Tou Ng, and Anthony Tung. 2021. Do Multi-Hop Question Answering Systems Know How to Answer the Single-Hop Sub-Questions?. In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume, pages 3244–3249, Online. Association for Computational Linguistics.
Cite (Informal):
Do Multi-Hop Question Answering Systems Know How to Answer the Single-Hop Sub-Questions? (Tang et al., EACL 2021)
Copy Citation:
PDF:
https://aclanthology.org/2021.eacl-main.283.pdf
Data
HotpotQAWikiHop