RetrievalRouter: Joint Modality and Architecture Selection for Document Retrieval

Emre Kuru, Mehmet Onur Keskin, Reza Farahbakhsh, Noel Crespi


Abstract
Document retrieval increasingly supports high-stakes information access in finance, healthcare, and law. Modern retrieval pipelines vary both in modality (text or multimodal) and in retrieval architecture (dense or late-interaction). These choices impose a hard compromise: the most effective pipelines are too slow and expensive to run at scale, while the fastest fail to retrieve evidence from complex documents. Practitioners must therefore choose between missed evidence and unusable latency, with no principled basis for adapting that choice at the query level. We show that this compromise is unnecessary. Not every query requires the same pipeline. Across benchmarks spanning financial and scientific corpora, no static pipeline dominates. We introduce RetrievalRouter, a lightweight query-aware router that learns, from the query text alone, which retrieval pipeline best fits each query. A single tunable parameter exposes the full accuracy–latency frontier, and for every static baseline, RetrievalRouter offers an operating point that is simultaneously more accurate and faster. Against the best static baseline, RetrievalRouter is 2.5% more accurate and 12.4× faster. Furthermore, compared with prior adaptive strategy selection methods, RetrievalRouter achieves significantly higher nDCG@5 across accuracy-oriented settings, while matching or numerically outperforming them on both nDCG@5 and latency in latency-oriented settings. Our code: https://github.com/emrekuruu/retrieval-router
Anthology ID:
2026.emnlp-main.1580
Volume:
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Month:
October
Year:
2026
Address:
Budapest, Hungary
Editors:
Hung-Yi Lee, Sunipa Dev, Jessy Li, Desmond Elliott
Venue:
EMNLP
SIG:
SIGDAT
Publisher:
Association for Computational Linguistics
Note:
Pages:
35968–35982
Language:
URL:
https://aclanthology.org/2026.emnlp-main.1580/
DOI:
Bibkey:
Cite (ACL):
Emre Kuru, Mehmet Onur Keskin, Reza Farahbakhsh, and Noel Crespi. 2026. RetrievalRouter: Joint Modality and Architecture Selection for Document Retrieval. In Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing, pages 35968–35982, Budapest, Hungary. Association for Computational Linguistics.
Cite (Informal):
RetrievalRouter: Joint Modality and Architecture Selection for Document Retrieval (Kuru et al., EMNLP 2026)
Copy Citation:
PDF:
https://aclanthology.org/2026.emnlp-main.1580.pdf