Declan Kutscher
Author directory2026
Tinted Frames: Question Framing Blinds Vision-Language Models
Wan-Cyuan Fan | Jiayun Luo | Declan Kutscher | Leonid Sigal | Ritwik Gupta
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Wan-Cyuan Fan | Jiayun Luo | Declan Kutscher | Leonid Sigal | Ritwik Gupta
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Vision-Language Models (VLMs) have been shown to be blind, often underutilizing visual inputs even on tasks that require visual reasoning. In this work, we demonstrate that VLMs are selectively blind. They modulate the attention to visual inputs based on linguistic framing even when alternative framings demand identical visual reasoning. Using visual attention as a probe, we quantify how framing alters both the amount and distribution of attention over the image. Constrained framings, such as multiple choice and yes/no, induce substantially lower attention to image context compared to open-ended, reduce focus on task-relevant regions, and shift attention towards uninformative tokens. We further demonstrate that this attention misallocation is a key mechanism contributing to degraded accuracy and cross-framing inconsistency. Building on this mechanism, we evaluate whether common prompting strategies mitigate framing-induced blindness. We find that simple image-focused instructions provide limited benefit, while structured reasoning and grounding prompts partially restore target-region attention and reduce, but do not eliminate, cross-framing inconsistency. Our findings suggest that question framing and prompting should be considered part of VLM evaluation to better reveal models’ visual capability.