DocHop-QA: Towards Multi-Hop Reasoning over Multimodal Document Collections

Jiwon Park, Seohyun Pyeon, Jinwoo Kim, Rina Carines Cabral, Zhenyuan He, Yihao Ding, Caren Han


Abstract
Despite rapid progress in large language models (LLMs), current QA benchmarks still overlook the core challenge of real-world scientific information seeking: synthesizing multimodal evidence scattered across multiple documents and structural formats. Existing QAs remain narrow in scope, relying on unimodal text and short-span reasoning that fail to capture the complexity of real information-seeking. We introduce DocHop-QA, a benchmark of 11,379 instances for evaluating multimodal, multi-document, multi-hop scientific QA. Built from publicly available PubMed articles, DocHop-QA incorporates textual passages, tables, and layout cues, enabling cross-document inference without explicit hyperlinks. To scale realistic QA construction, we develop an LLM-driven generation pipeline grounded in 11 scientific reasoning concepts, producing diverse and coherent question–answer pairs. To highlight the utility and versatility of the dataset, we propose a task-driven evaluation framework spanning four settings, including generative answering, multimodal evidence integration and structured index prediction. Experiments show that current models struggle with DocHop-QA’s long-context, multi-evidence demands, establishing it as a rigorous testbed for advancing next-generation scientific QA systems.
Anthology ID:
2026.emnlp-main.2563
Volume:
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Month:
October
Year:
2026
Address:
Budapest, Hungary
Editors:
Hung-Yi Lee, Sunipa Dev, Jessy Li, Desmond Elliott
Venue:
EMNLP
SIG:
SIGDAT
Publisher:
Association for Computational Linguistics
Note:
Pages:
58233–58261
Language:
URL:
https://aclanthology.org/2026.emnlp-main.2563/
DOI:
Bibkey:
Cite (ACL):
Jiwon Park, Seohyun Pyeon, Jinwoo Kim, Rina Carines Cabral, Zhenyuan He, Yihao Ding, and Caren Han. 2026. DocHop-QA: Towards Multi-Hop Reasoning over Multimodal Document Collections. In Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing, pages 58233–58261, Budapest, Hungary. Association for Computational Linguistics.
Cite (Informal):
DocHop-QA: Towards Multi-Hop Reasoning over Multimodal Document Collections (Park et al., EMNLP 2026)
Copy Citation:
PDF:
https://aclanthology.org/2026.emnlp-main.2563.pdf