Zeyu Wang
Author directoryOther people with similar names: Zeyu Wang, Zeyu Wang
Unverified author pages with similar names: Zeyu Wang
2026
Knowing Isn’t Always Saying: When Do Spatial Encodings Reach Answers in Vision-Language Models?
Zeyu Wang | Xinming Xu
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Zeyu Wang | Xinming Xu
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Vision-language models are known to encode spatial information in their hidden states, yet often fail to use it when answering. However, it remains unclear when and where this encoded information reaches the answer. We address this with direction patching, a class-conditioned causal intervention applied across layers, token positions, and prompt formats. Using spatial-ID directions constructed following prior encoding evidence, we find that causal influence on answer logits emerges only at mid-to-deep depths. Text chain-of-thought suppresses immediate object-word argmax-level transport in most models, while visually grounded prompts keep it open. Positive target-logit gain can remain below the argmax threshold, and transport can re-emerge at the final prefix token or at the answer step in deeper layers. Across the ten VLMs we study, these local effects form descriptive transport patterns. Complementary experiments characterize how these patterns shift across datasets, attributes, and encoding amplitudes. Together, these results reframe the encoding-grounding gap as a problem of conditional transport in VLMs.