Enhancing Generative Retrieval with Reinforcement Learning from Relevance Feedback

Yujia Zhou, Zhicheng Dou, Ji-Rong Wen


Abstract
The recent advent of end-to-end generative retrieval marks a significant shift in document retrieval methods, leveraging differentiable search indexes to directly produce relevant document identifiers (docids) in response to a specific query. Nevertheless, this approach faces two fundamental challenges: (i) a discrepancy between the token-level probabilistic optimization and the broader document-level relevance estimation; (ii) an overemphasis on top-1 results at the expense of overall ranking quality. To tackle these challenges, we propose a generative retrieval model with reinforcement learning from relevance feedback, which aims to align token-level docid generation with document-level relevance estimation. The training process incorporates three stages: supervised fine-tuning, relevance reward model training, and reinforced learning-to-rank from relevance feedback. To train a high-quality reward model, we define “relevance” under three progressive scenarios, which collectively offer a comprehensive evaluation of the document relevance. Experiments conducted on two benchmark datasets demonstrate the effectiveness of our proposed approach.
Anthology ID:
2023.emnlp-main.768
Volume:
Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing
Month:
December
Year:
2023
Address:
Singapore
Editors:
Houda Bouamor, Juan Pino, Kalika Bali
Venue:
EMNLP
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
12481–12490
Language:
URL:
https://aclanthology.org/2023.emnlp-main.768
DOI:
10.18653/v1/2023.emnlp-main.768
Bibkey:
Cite (ACL):
Yujia Zhou, Zhicheng Dou, and Ji-Rong Wen. 2023. Enhancing Generative Retrieval with Reinforcement Learning from Relevance Feedback. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 12481–12490, Singapore. Association for Computational Linguistics.
Cite (Informal):
Enhancing Generative Retrieval with Reinforcement Learning from Relevance Feedback (Zhou et al., EMNLP 2023)
Copy Citation:
PDF:
https://aclanthology.org/2023.emnlp-main.768.pdf
Video:
 https://aclanthology.org/2023.emnlp-main.768.mp4