Local Evidence, Global Credit: Visual Evidence-Calibrated Policy Optimization for Multimodal Reasoning

Zijing He, Zhongyi Deng, Kankan Lan, C.L.Philip Chen, Zeyu Chen, Enlei Gong, Tong Zhang


Abstract
Reinforcement Learning with Verifiable Rewards (RLVR) has substantially improved the reasoning capability of large vision-language models (LVLMs). Most RLVR methods for LVLMs follow an outcome-driven paradigm, using the correctness of final answer as supervision. However, in multimodal reasoning, this supervision overlooks a critical question: whether a successful reasoning trajectory is actually grounded in visual evidence. To address this limitation, we propose Visual Evidence-Calibrated Policy Optimization (VEPO), which turns visual grounding into an explicit trajectory-level credit calibration signal. VEPO first measures token-level visual responses by comparing the model’s likelihood under the original and perturbed visual inputs. It then estimates a visual background for the rollout-group and retains responses above the background as sparse visual anchors into a rollout-level evidence score. This score calibrates outcome-driven optimization and guides the policy toward successful trajectories with stronger visual support. Experiments on six multimodal reasoning benchmarks show that VEPO achieves strong overall performance against RLVR baselines without requiring additional annotations, demonstrating its scalability for multimodal policy optimization.
Anthology ID:
2026.findings-emnlp.1818
Volume:
Findings of the Association for Computational Linguistics: EMNLP 2026
Month:
October
Year:
2026
Address:
Budapest, Hungary
Editors:
Hung-Yi Lee, Sunipa Dev, Jessy Li, Desmond Elliott
Venue:
Findings
SIG:
SIGDAT
Publisher:
Association for Computational Linguistics
Note:
Pages:
38844–38858
Language:
URL:
https://aclanthology.org/2026.findings-emnlp.1818/
DOI:
10.18653/2026.findings-emnlp.1818
Bibkey:
Cite (ACL):
Zijing He, Zhongyi Deng, Kankan Lan, C.L.Philip Chen, Zeyu Chen, Enlei Gong, and Tong Zhang. 2026. Local Evidence, Global Credit: Visual Evidence-Calibrated Policy Optimization for Multimodal Reasoning. In Findings of the Association for Computational Linguistics: EMNLP 2026, pages 38844–38858, Budapest, Hungary. Association for Computational Linguistics.
Cite (Informal):
Local Evidence, Global Credit: Visual Evidence-Calibrated Policy Optimization for Multimodal Reasoning (He et al., Findings 2026)
Copy Citation:
PDF:
https://aclanthology.org/2026.findings-emnlp.1818.pdf