Hierarchical Representations in Dense Passage Retrieval for Question-Answering

Philipp Ennen, Federica Freddi, Chyi-Jiunn Lin, Po-Nien Kung, RenChu Wang, Chien-Yi Yang, Da-shan Shiu, Alberto Bernacchia


Abstract
An approach to improve question-answering performance is to retrieve accompanying information that contains factual evidence matching the question. These retrieved documents are then fed into a reader that generates an answer. A commonly applied retriever is dense passage retrieval. In this retriever, the output of a transformer neural network is used to query a knowledge database for matching documents. Inspired by the observation that different layers of a transformer network provide rich representations with different levels of abstraction, we hypothesize that useful queries can be generated not only at the output layer, but at every layer of a transformer network, and that the hidden representations of different layers may combine to improve the fetched documents for reader performance. Our novel approach integrates retrieval into each layer of a transformer network, exploiting the hierarchical representations of the input question. We show that our technique outperforms prior work on downstream tasks such as question answering, demonstrating the effectiveness of our approach.
Anthology ID:
2023.fever-1.2
Volume:
Proceedings of the Sixth Fact Extraction and VERification Workshop (FEVER)
Month:
May
Year:
2023
Address:
Dubrovnik, Croatia
Editors:
Mubashara Akhtar, Rami Aly, Christos Christodoulopoulos, Oana Cocarascu, Zhijiang Guo, Arpit Mittal, Michael Schlichtkrull, James Thorne, Andreas Vlachos
Venue:
FEVER
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
17–28
Language:
URL:
https://aclanthology.org/2023.fever-1.2
DOI:
10.18653/v1/2023.fever-1.2
Bibkey:
Cite (ACL):
Philipp Ennen, Federica Freddi, Chyi-Jiunn Lin, Po-Nien Kung, RenChu Wang, Chien-Yi Yang, Da-shan Shiu, and Alberto Bernacchia. 2023. Hierarchical Representations in Dense Passage Retrieval for Question-Answering. In Proceedings of the Sixth Fact Extraction and VERification Workshop (FEVER), pages 17–28, Dubrovnik, Croatia. Association for Computational Linguistics.
Cite (Informal):
Hierarchical Representations in Dense Passage Retrieval for Question-Answering (Ennen et al., FEVER 2023)
Copy Citation:
PDF:
https://aclanthology.org/2023.fever-1.2.pdf
Video:
 https://aclanthology.org/2023.fever-1.2.mp4