ShuaiHang Sun
Author directory2026
Binned Relative Policy Optimization towards Efficient Thinking of VLMs
ShuaiHang Sun | Zongzhao Li | Kai Shi | Jun Yang | Zhenyu Yang | Haonan Lu | Wenbing Huang
Findings of the Association for Computational Linguistics: EMNLP 2026
ShuaiHang Sun | Zongzhao Li | Kai Shi | Jun Yang | Zhenyu Yang | Haonan Lu | Wenbing Huang
Findings of the Association for Computational Linguistics: EMNLP 2026
Recent VLMs improve multimodal reasoning by generating explicit chains of thought, but often overthink with redundant reasoning. Although conventional length penalties mitigate this redundancy, they rely on coarse global rewards that fail to discern intermediate process quality or adapt to task complexity. To address this dilemma, we propose Binned Relative Policy Optimization (BRPO), a training-efficient framework that enables non-trivial process-level evaluation without introducing costly process reward models. Specifically, BRPO leverages high entropy lexical markers to segment reasoning trajectories into discrete intermediate prefixes, evaluating their problem-solving potential through forced answer completions. Building on this, we introduce Bin Normalization along with a length-aware adjustment to map prefixes into process bins, dynamically calibrating rewards and regulating length constraints based on reasoning stages and visual complexity. Experiments on six multimodal math benchmarks show that our method improves both accuracy and efficiency: on Qwen2.5-VL-7B, it raises average accuracy from 47.8% to 53.6% while reducing the average response length by 39.0%.