Qingheng Xiong
Author directory2026
DR3-Eval: Towards Realistic and Reproducible Deep Research Evaluation
Qianqian Xie | He Zhu | Xueming Han | Qingheng Xiong | Tiantian Xia | Fanyu Meng | Jiakai Wang | Zhiqi Bai | Chengkang Jiang | Zhaohui Wang | Yubin Guo | Yuqing Wen | Jiayang Mao | Zijie Zhang | Shihao Li | Yanghai Wang | Yuxiang Ren | Junlan Feng | Jiaheng Liu
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Qianqian Xie | He Zhu | Xueming Han | Qingheng Xiong | Tiantian Xia | Fanyu Meng | Jiakai Wang | Zhiqi Bai | Chengkang Jiang | Zhaohui Wang | Yubin Guo | Yuqing Wen | Jiayang Mao | Zijie Zhang | Shihao Li | Yanghai Wang | Yuxiang Ren | Junlan Feng | Jiaheng Liu
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Deep Research Agents (DRAs) aim to solve complex, long-horizon research tasks involving planning, retrieval, multimodal understanding, and report generation, yet their evaluation remains challenging due to dynamic web environments and ambiguous task definitions. We propose **DR3-Eval**, a realistic and reproducible benchmark for evaluating deep research agents on multimodal, multi-file report generation. DR3-Eval is constructed from authentic user-provided materials and paired with a per-task static research sandbox corpus that simulates open-web complexity while remaining fully verifiable, containing supportive documents, distractors, and noise. Moreover, we introduce a multi-dimensional evaluation framework measuring Information Recall, Factual Accuracy, Citation Coverage, Instruction Following, and Depth Quality, and validate its alignment with human judgments. Experiments with our developed multi-agent system DR3-Agent based on multiple state-of-the-art language models demonstrate that DR3-Eval is highly challenging and reveals critical failure modes in retrieval robustness and hallucination control.