PROBE: Prospective Retrieval with Oracle-Based End-Goal Distillation for Multi-Hop Question Answering

Xu Lu, Zhengqing Zang, Binhui Yao, Luwei Yang, Ruimin Chen, Zeng Wang, Haobo Wang


Abstract
Multi-hop question answering (MHQA) requires models to retrieve and reason over evidence distributed across multiple documents. However, training dense retrievers for MHQA remains challenging because labeled query-document pairs are difficult to obtain for the evolving intermediate queries in multi-hop reasoning. Recent approaches use LLM likelihoods for step-wise supervision over queries conditioned on accumulated evidence, which anchors retrieval to known facts and biases supervision toward answer-bearing documents, leaving bridging evidence undiscovered. To address this, we propose PROBE, a framework that optimizes dense retrievers for MHQA without requiring manually annotated query-document pairs. It decouples the reasoning state into a factual summary and a prospective query that anticipates missing information. For training, we transfer cross-encoder signals only at the final hop, providing trajectory-level supervision without penalizing early exploration. Experiments on three MHQA benchmarks show that PROBE consistently improves both retrieval and QA performance over strong iterative baselines, achieving an average gain of 5.7 points while using only about 5% of the training data of the prior method.
Anthology ID:
2026.emnlp-main.1379
Volume:
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Month:
October
Year:
2026
Address:
Budapest, Hungary
Editors:
Hung-Yi Lee, Sunipa Dev, Jessy Li, Desmond Elliott
Venue:
EMNLP
SIG:
SIGDAT
Publisher:
Association for Computational Linguistics
Note:
Pages:
31494–31515
Language:
URL:
https://aclanthology.org/2026.emnlp-main.1379/
DOI:
Bibkey:
Cite (ACL):
Xu Lu, Zhengqing Zang, Binhui Yao, Luwei Yang, Ruimin Chen, Zeng Wang, and Haobo Wang. 2026. PROBE: Prospective Retrieval with Oracle-Based End-Goal Distillation for Multi-Hop Question Answering. In Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing, pages 31494–31515, Budapest, Hungary. Association for Computational Linguistics.
Cite (Informal):
PROBE: Prospective Retrieval with Oracle-Based End-Goal Distillation for Multi-Hop Question Answering (Lu et al., EMNLP 2026)
Copy Citation:
PDF:
https://aclanthology.org/2026.emnlp-main.1379.pdf