AdaptiveDocRAG: Efficient Document Understanding with Multimodal Large Language Models

Surya Selvam, Anand Raghunathan


Abstract
Document understanding requires answering questions over long, multi-page documents that interleave text, figures, and tables. Recent methods adopt the Document Retrieval-Augmented Generation (DocRAG) paradigm, which indexes each document and feeds the top-K most relevant pages to a Multimodal Large Language Model (MLLM) per query. Because each page can be represented as extracted text or a rendered image, DocRAG systems expose two key knobs: an indexing modality choice when the documents are ingested and a context modality at query time. Existing systems either fix both knobs to the visual modality or use both modalities throughout, incurring indexing and query costs regardless of document or query type. We propose AdaptiveDocRAG, which adapts both knobs as well as the size of visual context to optimize the trade-off between accuracy, query response time, and indexing time. AdaptiveDocRAG achieves this through three lightweight modules. First, document-adaptive indexing modality selection uses a text-density heuristic to route text-heavy documents to cheaper textual indexing. Second, query-adaptive context modality selection uses a learned per-query router to select textual or visual context. Third, query-aware visual context compression reuses the retriever’s per-patch query-page similarity to crop relevant sub-page regions and fuse them with top-ranked text chunks. On MMLongBench-Doc and NQ-Text with Qwen3-VL-8B, AdaptiveDocRAG matches or improves visual-only DocRAG accuracy (up to +1.1 points) at ~50% of the MLLM tokens, with 1.30-1.79x end-to-end speedup and 1.34x faster indexing.
Anthology ID:
2026.docinsights-1.5
Volume:
The Proceedings of Workshop on Document Intelligence and Understanding
Month:
October
Year:
2026
Address:
Budapest, Hungary
Editors:
Santosh T.Y.S.S., Amit Agarwal, Vivek Gupta, Yova Kementchedjhieva, Aparna Garimella, Yufang Hou
Venues:
DocInsights | WS
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
63–72
Language:
URL:
https://aclanthology.org/2026.docinsights-1.5/
DOI:
Bibkey:
Cite (ACL):
Surya Selvam and Anand Raghunathan. 2026. AdaptiveDocRAG: Efficient Document Understanding with Multimodal Large Language Models. In The Proceedings of Workshop on Document Intelligence and Understanding, pages 63–72, Budapest, Hungary. Association for Computational Linguistics.
Cite (Informal):
AdaptiveDocRAG: Efficient Document Understanding with Multimodal Large Language Models (Selvam & Raghunathan, DocInsights 2026)
Copy Citation:
PDF:
https://aclanthology.org/2026.docinsights-1.5.pdf