Zhi Gao
Author directory2026
AdaptMMBench: Benchmarking Adaptive Multimodal Reasoning for Mode Selection and Reasoning Process
Xintong Zhang | Xiaowen Zhang | Jingrong Wu | Zhi Gao | Shilin Yan | Zhenxin Diao | Kunpeng Gao | Xuanyan Chen | Yuwei Wu | Yunde Jia | Qing Li
Findings of the Association for Computational Linguistics: EMNLP 2026
Xintong Zhang | Xiaowen Zhang | Jingrong Wu | Zhi Gao | Shilin Yan | Zhenxin Diao | Kunpeng Gao | Xuanyan Chen | Yuwei Wu | Yunde Jia | Qing Li
Findings of the Association for Computational Linguistics: EMNLP 2026
Adaptive multimodal reasoning has emerged as a promising frontier in Vision-Language Models (VLMs), aiming to dynamically modulate between tool-augmented visual reasoning and text reasoning to enhance both effectiveness and efficiency. However, existing evaluations rely on static difficulty labels and simple metrics, overlooking the model-dependent nature of task difficulties: a task that is challenging for one model may be trivial for another. Consequently, they obscure the distinction between reasoning mode selection and overall task accuracy while neglecting fine-grained reasoning process analyses. In this paper, we propose AdaptMMBench, a comprehensive benchmark for adaptive multimodal reasoning across five domains: natural scene, OCR, GUI, knowledge, and math, encompassing both direct perception and complex reasoning tasks. AdaptMMBench utilizes a Matthews Correlation Coefficient (MCC) metric to evaluate the quality of reasoning mode selection, isolating this meta-cognition ability by dynamically identifying task difficulties based on models’ capability boundaries. Moreover, AdaptMMBench facilitates multi-dimensional reasoning process evaluation across key step coverage, tool effectiveness, and computational efficiency. Our evaluation shows that strong reasoning mode selection is helpful, but final adaptive gains and efficiency also depend on overall model capability. Process metrics further diagnose bottlenecks in reasoning coherence, tool use, and cost for improving adaptive performance and efficiency.
Reliability-Prioritized Fine-Grained Generation in Multimodal Large Language Models
Xiaomeng Fan | Wei Wu | Yuwei Wu | Zhi Gao | Luo Shiyu | Mingyang Gao | Haoyu Zhao | Zhenxin Diao | Yuxuan Ba | Lijia Feng | Yunde Jia | Mehrtash Harandi
Findings of the Association for Computational Linguistics: EMNLP 2026
Xiaomeng Fan | Wei Wu | Yuwei Wu | Zhi Gao | Luo Shiyu | Mingyang Gao | Haoyu Zhao | Zhenxin Diao | Yuxuan Ba | Lijia Feng | Yunde Jia | Mehrtash Harandi
Findings of the Association for Computational Linguistics: EMNLP 2026
Multimodal large language models (MLLMs) are increasingly expected to generate fine-grained descriptions of visual content. However, we observe and theoretically show that generating fine-grained responses poses a reliability challenge, i.e., fine-grained generation is more error-prone than coarse-grained generation. This phenomenon suggests that models should generate the finest description that remains reliable rather than simply produce more specific outputs. To investigate this problem, we develop GranFact, a granularity-aware benchmark consisting of expert-verified multi-object images with coarse-to-fine category annotations. Then, we design a hierarchy-aware evaluation algorithm, which assesses both whether model predictions are visually correct and how specific the correct predictions are. We also propose a reliability-prioritized preference optimization method based on Direct Preference Optimization (DPO), which penalizes unreliable fine-grained claims while rewarding reliable specificity. Experiments on GranFact show that our method improves fine-grained generation while preserving reliability. Code and data are available at https://github.com/WeiWu2025/GranFact.
Bridging Modality Disconnect in Self-Reflection via Closed-Loop Visually Grounded Verification
Haoyu Zhang | Zhi Gao | Pengxiang Li | Xintong Zhang | Yuwei Wu | Rui Gao | Mingyang Gao | Xuanyan Chen | Anqian Cen | Che Sun | Yunde Jia
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Haoyu Zhang | Zhi Gao | Pengxiang Li | Xintong Zhang | Yuwei Wu | Rui Gao | Mingyang Gao | Xuanyan Chen | Anqian Cen | Che Sun | Yunde Jia
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Self-reflection has become a key mechanism for improving reasoning in Vision-Language Models (VLMs), yet this corrective mechanism often fails when resolving complex fine-grained visual ambiguities. This performance degradation stems from the issue of modality disconnect in self-reflection: most existing models execute self-reflection either within textual or latent space, lacking an explicit mechanism to align textual reasoning with visual evidence. In this paper, we propose MIRROR, a closed-loop visual reflection framework comprising four steps: response generation, error identification, region-based visual verification, and revision. In this cycle, the model first generates an initial response, identifies uncertain logical statements in the response that require visual verification, then grounds them in relevant image regions, and finally revises based on the visual evidence. We construct a multi-turn visual reflection dataset ReflectV, which empowers the model with such a reflective capability. Extensive experiments across diverse multimodal benchmarks show that MIRROR improves performance and reduces visual hallucinations by an average of about 7.2%, demonstrating the advantage of transforming self-reflection from open-loop textual revision into closed-loop, visually grounded verification.
Fully Decoupled Reinforcement Learning with Adaptive Data Curation for GUI Agents
Pengxiang Li | Zechenhu | Zirui Shang | Jingrong Wu | Yang Liu | Hui Liu | Zhi Gao | Chenrui Shi | Bofei Zhang | Zihao Zhang | Xiaochuan Shi | Zedong YU | Yuwei Wu | Xinxiao Wu | Yunde Jia | Liuyu Xiang | Zhaofeng He | Qing Li
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Pengxiang Li | Zechenhu | Zirui Shang | Jingrong Wu | Yang Liu | Hui Liu | Zhi Gao | Chenrui Shi | Bofei Zhang | Zihao Zhang | Xiaochuan Shi | Zedong YU | Yuwei Wu | Xinxiao Wu | Yunde Jia | Liuyu Xiang | Zhaofeng He | Qing Li
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Vision-Language Models (VLMs) have catalyzed the development of Graphical User Interface (GUI) agents for desktop automation. However, reinforcement learning (RL) for GUI agents remains bottlenecked by two challenges: existing RL frameworks still leave substantial idle time inside the rollout stage, and the learning signal from long-horizon, sparse-reward interactions is highly imbalanced. In this paper, we introduce DART, a fully Decoupled Agentic RL Training framework for GUI agents. Whereas prior RL designs only separate the trainer from the rollout, DART splits the pipeline into four independent modules: environment cluster, rollout service, data manager, and trainer, which brings two extra efficiency benefits: rollout-wise trajectory scheduling and per-worker model synchronization. This yields a 1.6× increase in GPU utilization, a 1.9× boost in training throughput, and a 5.5× increase in environment utilization over a non-decoupled baseline. We further propose a multi-level adaptive data curation scheme operating at the task, trajectory, step, and token levels: it reallocates rollout budget by task difficulty, bootstraps hard tasks with a pre-collected experience pool, restricts policy updates to high-entropy critical steps, and applies truncated importance sampling to absorb the distribution gap between th rollout and training engines. On the OSWorld benchmark, our DART-GUI-7B achieves a 42.13 task success rate, a 14.61 absolute improvement over its base model and a new state of the art among open-source 7B-scale agents. Code, data, and model checkpoints are open-sourced at https://dart-gui-anonymous.github.io (anonymous).
Search
Fix author
Co-authors
- Yunde Jia 4
- Yuwei Wu 4
- Xuanyan Chen 2
- Zhenxin Diao 2
- Mingyang Gao 2
- Pengxiang Li 2
- Qing Li 2
- Jingrong Wu 2
- Xintong Zhang 2
- Yuxuan Ba 1
- Anqian Cen 1
- Xiaomeng Fan 1
- Lijia Feng 1
- Kunpeng Gao 1
- Rui Gao 1
- Mehrtash Harandi 1
- Zhaofeng He 1
- Hui Liu 1
- Yang Liu 1
- Zirui Shang 1
- Chenrui Shi 1
- Xiaochuan Shi 1
- Luo Shiyu 1
- Che Sun 1
- Wei Wu 1
- Xinxiao Wu 1
- Liuyu Xiang 1
- Zedong YU 1
- Shilin Yan 1
- Zechenhu 1
- Bofei Zhang 1
- Haoyu Zhang 1
- Xiaowen Zhang 1
- Zihao Zhang 1
- Haoyu Zhao 1