EVO-RAG: Evolving Retrieval-Augmented Agents for Efficient Multi-Hop Query Optimization

Yuelyu Ji, Rui Meng, Zhuochun Li, Daqing He


Abstract
Retrieval-augmented generation (RAG) grounds large language models (LLMs) in external evidence, yet multi-hop pipelines still suffer from redundant sub-queries, shallow exploration, and premature or delayed stopping. We present EVO-RAG, a phase-aware framework that couples a lightweight two-stage curriculum (Discovery→Refinement) with seven step-level rewards and an in-episode time scheduler. The scheduler decays exploration incentives as evidence accumulates while increasing efficiency and correctness pressure as uncertainty shrinks. Beyond scalar rewards, we train a multi-head preference model and benchmark DPO, PPO, and GRPO under identical rollouts and curricula for a controlled comparison. Evaluated on HotpotQA, 2WikiMultiHopQA, MuSiQue, and Bamboogle with 8B-class backbones, EVO-RAG improves EM/F1 while reducing redundant hops. Ablations show that suppressing query overlap, rewarding controlled backtracking and justified refusal, and time-dynamic weighting are key to the accuracy–efficiency trade-off.
Anthology ID:
2026.groundlm-1.6
Volume:
Proceedings of the 1st Workshop on Grounding Language Models: Learning Faithfully and Efficiently (GroundLM 2026)
Month:
October
Year:
2026
Address:
Budapest, Hungary
Editors:
Yimu Wang, Yee Man Choi, Di Wu, Siqi Ouyang, Mozhgan Nasr Azadani, Yi R. (May) Fung
Venues:
GroundLM | WS
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
72–83
Language:
URL:
https://aclanthology.org/2026.groundlm-1.6/
DOI:
Bibkey:
Cite (ACL):
Yuelyu Ji, Rui Meng, Zhuochun Li, and Daqing He. 2026. EVO-RAG: Evolving Retrieval-Augmented Agents for Efficient Multi-Hop Query Optimization. In Proceedings of the 1st Workshop on Grounding Language Models: Learning Faithfully and Efficiently (GroundLM 2026), pages 72–83, Budapest, Hungary. Association for Computational Linguistics.
Cite (Informal):
EVO-RAG: Evolving Retrieval-Augmented Agents for Efficient Multi-Hop Query Optimization (Ji et al., GroundLM 2026)
Copy Citation:
PDF:
https://aclanthology.org/2026.groundlm-1.6.pdf