Other people with similar names: Xin Yu

Unverified author pages with similar names: Xin Yu


2026

MLLM-based GUI grounding methods commonly formulate target localization as autoregressive coordinate generation, enabling models to leverage strong instruction following and semantic understanding. However, by casting localization as coordinate-token prediction, these methods rely on implicit decoding to convert region-level target evidence into precise click coordinates, making point-level spatial alignment unreliable. Through diagnostic analysis, we find that intermediate decoder layers already exhibit target-region awareness, yet such region-level evidence fails to guide final point-level decisions. To address this, we propose InnerZoom, a single-forward cross-layer evidence bridging framework that refines target evidence inside the model during decoding, avoiding extra inferences required by Zoom-In. InnerZoom reframes coordinate decoding as an evidence-preserving process by identifying target-related cues within the original forward pass, refining them across decoder layers, and reinjecting the refined evidence into coordinate prediction. Extensive experiments show that InnerZoom-4B achieves state-of-the-art performance on all six mainstream GUI grounding benchmarks, outperforming the second-best result by 4.1 points on OSWorld-G.Under the 4B setting, InnerZoom improves the same SFT+RL baseline by 5.3 points on average and outperforms two-pass Zoom-In by 1.3 points on average, while reducing measured GPU inference latency by 44% and estimated TFLOPs by 29%. Code and models will be at https://github.com/YenanLiu/InnerZoom.

2024

Large Multimodal Models (LMMs) have achieved strong performance across a range of vision and language tasks. However, their spatial reasoning capabilities are under-investigated. In this paper, we construct a novel VQA dataset, Spatial-MM, to comprehensively study LMMs’ spatial understanding and reasoning capabilities. Our analyses on object-relationship and multi-hop reasoning reveal several important findings. Firstly, bounding boxes and scene graphs, even synthetic ones, can significantly enhance LMMs’ spatial reasoning. Secondly, LMMs struggle more with questions posed from the human perspective than the camera perspective about the image. Thirdly, chain of thought (CoT) prompting does not improve model performance on complex multi-hop questions involving spatial relations. Moreover, spatial reasoning steps are much less accurate than non-spatial ones across MLLMs. Lastly, our perturbation analysis on GQA-spatial reveals that LMMs are much stronger at basic object detection than complex spatial reasoning. We believe our new benchmark dataset and in-depth analyses can spark further research on LMMs spatial reasoning.