APEX-Searcher: Hierarchical Credit Assignment with Subgoaling for Agentic Retrieval-Augmented Generation

Kun Chen, Qingchao Kong, Zhao Feifei, Wenji Mao


Abstract
Retrieval-augmented generation (RAG) connects large language models (LLMs) to external knowledge, but single-round retrieval is often insufficient for complex multi-hop questions. To improve search on complex tasks, most existing works combine multi-round retrieval with reasoning through end-to-end training. While these approaches improve problem-solving performance, they still face challenges in task reasoning and model training. First, due to the lack of an explicit subgoaling mechanism, most existing RAG methods may produce ambiguous retrieval execution paths, leading to the degradation of model performance. Second, a few existing RAG methods have started to incorporate planning and execution into the model training process. However, they typically rely on a single final reward, causing hierarchical credit entanglement between planning and execution and preventing the model from clearly separating plan errors from retrieval errors. To address these limitations, we propose APEX-Searcher, a coordinated planner–executor framework that optimizes a recursive planner with RL to accommodate multiple valid, dependency-aware decompositions while training the executor with SFT under clearer subtask-level supervision. Extensive experiments show consistent gains in both multi-hop RAG and task planning across benchmarks.
Anthology ID:
2026.findings-emnlp.1538
Volume:
Findings of the Association for Computational Linguistics: EMNLP 2026
Month:
October
Year:
2026
Address:
Budapest, Hungary
Editors:
Hung-Yi Lee, Sunipa Dev, Jessy Li, Desmond Elliott
Venue:
Findings
SIG:
SIGDAT
Publisher:
Association for Computational Linguistics
Note:
Pages:
32708–32726
Language:
URL:
https://aclanthology.org/2026.findings-emnlp.1538/
DOI:
Bibkey:
Cite (ACL):
Kun Chen, Qingchao Kong, Zhao Feifei, and Wenji Mao. 2026. APEX-Searcher: Hierarchical Credit Assignment with Subgoaling for Agentic Retrieval-Augmented Generation. In Findings of the Association for Computational Linguistics: EMNLP 2026, pages 32708–32726, Budapest, Hungary. Association for Computational Linguistics.
Cite (Informal):
APEX-Searcher: Hierarchical Credit Assignment with Subgoaling for Agentic Retrieval-Augmented Generation (Chen et al., Findings 2026)
Copy Citation:
PDF:
https://aclanthology.org/2026.findings-emnlp.1538.pdf