Subtraction Gets You More: Gap-Aware Retrieval for Multimodal Multi-Hop QA

Sunah O, Jay-Yoon Lee


Abstract
In multimodal multi-hop question answering, we focus on the initial retrieval stage via two distinct tasks: (1) evidence set completion, retrieving missing evidence given context, and (2) sequential pool construction, iteratively building the top-K pool from the scratch. Under these settings, we point out that conventional iterative retrieval frameworks often suffer from Semantic Anchoring, where previously fetched evidence traps the retriever and yields entity-centric redundancy. To break this trap, we propose GRAIL (Gap-aware Retrieval via Adaptive Implicit Localization), a paradigm that performs implicit query rewriting directly at the embedding level. By context-subtractive query steering, GRAIL excels at compositional cross-modal reasoning, while additive embedding updates show strength on localized information aggregation. To leverage these complementary strengths, we propose a Task-Adaptive Hybrid Framework, achieving a 35.7% macro-averaged performance gain on the MultimodalQA evidence set completion task. Extensive evaluations demonstrate that sequential GRAIL retrieves in a superior, noise-resilient manner, significantly expanding the search horizon through iterative gap-aware optimization.
Anthology ID:
2026.emnlp-main.2132
Volume:
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Month:
October
Year:
2026
Address:
Budapest, Hungary
Editors:
Hung-Yi Lee, Sunipa Dev, Jessy Li, Desmond Elliott
Venue:
EMNLP
SIG:
SIGDAT
Publisher:
Association for Computational Linguistics
Note:
Pages:
48472–48490
Language:
URL:
https://aclanthology.org/2026.emnlp-main.2132/
DOI:
Bibkey:
Cite (ACL):
Sunah O and Jay-Yoon Lee. 2026. Subtraction Gets You More: Gap-Aware Retrieval for Multimodal Multi-Hop QA. In Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing, pages 48472–48490, Budapest, Hungary. Association for Computational Linguistics.
Cite (Informal):
Subtraction Gets You More: Gap-Aware Retrieval for Multimodal Multi-Hop QA (O & Lee, EMNLP 2026)
Copy Citation:
PDF:
https://aclanthology.org/2026.emnlp-main.2132.pdf