Re3val: Reinforced and Reranked Generative Retrieval

EuiYul Song, Sangryul Kim, Haeju Lee, Joonkee Kim, James Thorne


Abstract
Generative retrieval models encode pointers to information in a corpus as an index within the model’s parameters. These models serve as part of a larger pipeline, where retrieved information conditions generation for knowledge-intensive NLP tasks. However, we identify two limitations: the generative retrieval does not account for contextual information. Secondly, the retrieval can’t be tuned for the downstream readers as decoding the page title is a non-differentiable operation. This paper introduces Re3val, trained with generative reranking and reinforcement learning using limited data. Re3val leverages context acquired via Dense Passage Retrieval to rerank the retrieved page titles and utilizes REINFORCE to maximize rewards generated by constrained decoding. Additionally, we generate questions from our pre-training dataset to mitigate epistemic uncertainty and bridge the domain gap between the pre-training and fine-tuning datasets. Subsequently, we extract and rerank contexts from the KILT database using the rerank page titles. Upon grounding the top five reranked contexts, Re3val demonstrates the Top 1 KILT scores compared to all other generative retrieval models across five KILT datasets.
Anthology ID:
2024.findings-eacl.27
Volume:
Findings of the Association for Computational Linguistics: EACL 2024
Month:
March
Year:
2024
Address:
St. Julian’s, Malta
Editors:
Yvette Graham, Matthew Purver
Venue:
Findings
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
393–409
Language:
URL:
https://aclanthology.org/2024.findings-eacl.27
DOI:
Bibkey:
Cite (ACL):
EuiYul Song, Sangryul Kim, Haeju Lee, Joonkee Kim, and James Thorne. 2024. Re3val: Reinforced and Reranked Generative Retrieval. In Findings of the Association for Computational Linguistics: EACL 2024, pages 393–409, St. Julian’s, Malta. Association for Computational Linguistics.
Cite (Informal):
Re3val: Reinforced and Reranked Generative Retrieval (Song et al., Findings 2024)
Copy Citation:
PDF:
https://aclanthology.org/2024.findings-eacl.27.pdf
Video:
 https://aclanthology.org/2024.findings-eacl.27.mp4