IndexRAG: Index-Time Reasoning for Multi-Hop Retrieval-Augmented Generation

Zhenghua Bao, Yi Shi


Abstract
Multi-hop question answering (QA) requires reasoning across multiple documents, yet existing retrieval-augmented generation (RAG) approaches address this either through graph-based methods requiring additional online processing or iterative multi-step reasoning. We present IndexRAG, a novel approach that shifts cross-document reasoning from online inference to offline indexing. IndexRAG identifies bridge entities shared across documents and generates bridging facts as independently retrievable units, requiring no additional training or fine-tuning. Experiments on three widely-used multi-hop QA benchmarks (HotpotQA, 2WikiMultiHopQA, MuSiQue) show that IndexRAG improves F1 over Naive RAG by 4.6 points on average, while requiring only single-pass retrieval and a single LLM call at inference time. When combined with IRCoT, IndexRAG achieves the best average performance among all evaluated methods, including graph-based baselines such as HippoRAG2 and FastGraphRAG, while relying on a flat vector index. Our code is available at https://github.com/Continuum-AI-Corp/IndexRAG.
Anthology ID:
2026.findings-aacl.175
Volume:
Findings of the Association for Computational Linguistics: IJCNLP-AACL 2026
Month:
November
Year:
2026
Address:
Hengqin, China
Editors:
Yue Zhang, Ruifeng Xu, Ayu Purwarianti, Tong Rong, Jun Suzuki
Venue:
Findings
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
3491–3506
Language:
URL:
https://aclanthology.org/2026.findings-aacl.175/
DOI:
Bibkey:
Cite (ACL):
Zhenghua Bao and Yi Shi. 2026. IndexRAG: Index-Time Reasoning for Multi-Hop Retrieval-Augmented Generation. In Findings of the Association for Computational Linguistics: IJCNLP-AACL 2026, pages 3491–3506, Hengqin, China. Association for Computational Linguistics.
Cite (Informal):
IndexRAG: Index-Time Reasoning for Multi-Hop Retrieval-Augmented Generation (Bao & Shi, Findings 2026)
Copy Citation:
PDF:
https://aclanthology.org/2026.findings-aacl.175.pdf