Evidentiality-aware Retrieval for Overcoming Abstractiveness in Open-Domain Question Answering

Yongho Song, Dahyun Lee, Myungha Jang, Seung-won Hwang, Kyungjae Lee, Dongha Lee, Jinyoung Yeo


Abstract
The long-standing goal of dense retrievers in abtractive open-domain question answering (ODQA) tasks is to learn to capture evidence passages among relevant passages for any given query, such that the reader produce factually correct outputs from evidence passages. One of the key challenge is the insufficient amount of training data with the supervision of the answerability of the passages. Recent studies rely on iterative pipelines to annotate answerability using signals from the reader, but their high computational costs hamper practical applications. In this paper, we instead focus on a data-driven approach and propose Evidentiality-Aware Dense Passage Retrieval (EADPR), which leverages synthetic distractor samples to learn to discriminate evidence passages from distractors. We conduct extensive experiments to validate the effectiveness of our proposed method on multiple abstractive ODQA tasks.
Anthology ID:
2024.findings-eacl.130
Volume:
Findings of the Association for Computational Linguistics: EACL 2024
Month:
March
Year:
2024
Address:
St. Julian’s, Malta
Editors:
Yvette Graham, Matthew Purver
Venue:
Findings
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
1930–1943
Language:
URL:
https://aclanthology.org/2024.findings-eacl.130
DOI:
Bibkey:
Cite (ACL):
Yongho Song, Dahyun Lee, Myungha Jang, Seung-won Hwang, Kyungjae Lee, Dongha Lee, and Jinyoung Yeo. 2024. Evidentiality-aware Retrieval for Overcoming Abstractiveness in Open-Domain Question Answering. In Findings of the Association for Computational Linguistics: EACL 2024, pages 1930–1943, St. Julian’s, Malta. Association for Computational Linguistics.
Cite (Informal):
Evidentiality-aware Retrieval for Overcoming Abstractiveness in Open-Domain Question Answering (Song et al., Findings 2024)
Copy Citation:
PDF:
https://aclanthology.org/2024.findings-eacl.130.pdf