Yang Yao
Author directoryOther people with similar names: Yang Yao
2026
VC-IF: Introducing Visual-Constraint Instruction Following for Multi-Turn, Multi-Constraint Vision-Language Models
Jixiang Hong | Pi Bu | Yingyao Wang | Bo Peng | Peijie Wang | Chen Wang | Tengtao Song | Yang Yao | Jun Song | Rui Yan | Cheng Yu | Bo Zheng
Findings of the Association for Computational Linguistics: EMNLP 2026
Jixiang Hong | Pi Bu | Yingyao Wang | Bo Peng | Peijie Wang | Chen Wang | Tengtao Song | Yang Yao | Jun Song | Rui Yan | Cheng Yu | Bo Zheng
Findings of the Association for Computational Linguistics: EMNLP 2026
Large vision-language models (LVLMs) have made rapid advancements; however, existing benchmarks for evaluating their instruction-following abilities focus on purely textual constraints, e.g., format constraints and role-playing, which assess only language-level adherence and are not related to vision. To bridge this gap, we introduce VC-IF, a benchmark that involves visual constraints connecting instructions to explicit operations over image content, thereby directly probing visual spatial understanding and compositional compliance. VC-IF defines atomic constraints, systematically combining mixed settings, scaling from single- to multi-constraint scenarios, and single- and multi-turn scenarios. We evaluate 16 popular LVLMs and find that models performing well under text-only constraints may struggle when faced with visual constraints, highlighting significant gaps in their capabilities. Further, we conduct scaling analyses across varying numbers of constraints and dialogue turns, revealing performance trends as constraint complexity increases along different dimensions. Finally, we synthesize a large training dataset and propose VC-PO, a reinforcement learning strategy that uses a hybrid verifier mechanism to compute reward signals for multiple constraints. Applying VC-PO to Qwen3-VL-8B-Instruct improves its overall VC-IF score from 43.74 to 55.18 (+11.44). The VC-PO-trained model remains stable as the number of turns and constraints increases. Our data and code are available at VC-IF.