Xin Yu
Author directoryOther people with similar names: Xin Yu
Unverified author pages with similar names: Xin Yu
2026
One Forward Beats Two: InnerZoom for Accurate and Efficient GUI Grounding
Chen Liu | Ling Chen | Hanzhang Zhou | Liangyu Chen | Chenglin Cai | Xin Yu | Steven Hoi | Yue Wang
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Chen Liu | Ling Chen | Hanzhang Zhou | Liangyu Chen | Chenglin Cai | Xin Yu | Steven Hoi | Yue Wang
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
MLLM-based GUI grounding methods commonly formulate target localization as autoregressive coordinate generation, enabling models to leverage strong instruction following and semantic understanding. However, by casting localization as coordinate-token prediction, these methods rely on implicit decoding to convert region-level target evidence into precise click coordinates, making point-level spatial alignment unreliable. Through diagnostic analysis, we find that intermediate decoder layers already exhibit target-region awareness, yet such region-level evidence fails to guide final point-level decisions. To address this, we propose InnerZoom, a single-forward cross-layer evidence bridging framework that refines target evidence inside the model during decoding, avoiding extra inferences required by Zoom-In. InnerZoom reframes coordinate decoding as an evidence-preserving process by identifying target-related cues within the original forward pass, refining them across decoder layers, and reinjecting the refined evidence into coordinate prediction. Extensive experiments show that InnerZoom-4B achieves state-of-the-art performance on all six mainstream GUI grounding benchmarks, outperforming the second-best result by 4.1 points on OSWorld-G.Under the 4B setting, InnerZoom improves the same SFT+RL baseline by 5.3 points on average and outperforms two-pass Zoom-In by 1.3 points on average, while reducing measured GPU inference latency by 44% and estimated TFLOPs by 29%. Code and models will be at https://github.com/YenanLiu/InnerZoom.
2024
An Empirical Analysis on Spatial Reasoning Capabilities of Large Multimodal Models
Fatemeh Shiri | Xiao-Yu Guo | Mona Golestan Far | Xin Yu | Reza Haf | Yuan-Fang Li
Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing
Fatemeh Shiri | Xiao-Yu Guo | Mona Golestan Far | Xin Yu | Reza Haf | Yuan-Fang Li
Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing
Large Multimodal Models (LMMs) have achieved strong performance across a range of vision and language tasks. However, their spatial reasoning capabilities are under-investigated. In this paper, we construct a novel VQA dataset, Spatial-MM, to comprehensively study LMMs’ spatial understanding and reasoning capabilities. Our analyses on object-relationship and multi-hop reasoning reveal several important findings. Firstly, bounding boxes and scene graphs, even synthetic ones, can significantly enhance LMMs’ spatial reasoning. Secondly, LMMs struggle more with questions posed from the human perspective than the camera perspective about the image. Thirdly, chain of thought (CoT) prompting does not improve model performance on complex multi-hop questions involving spatial relations. Moreover, spatial reasoning steps are much less accurate than non-spatial ones across MLLMs. Lastly, our perturbation analysis on GQA-spatial reveals that LMMs are much stronger at basic object detection than complex spatial reasoning. We believe our new benchmark dataset and in-depth analyses can spark further research on LMMs spatial reasoning.