Xiaoxiao Sun
Author directory2026
ScreenHaystack: Finding Blind Zones in GUI Grounding
Chenyue Li | Xiaoxiao Sun | Yubo Deng | Qinlin Zhao | Serena Yeung-Levy | Yuhui Zhang
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Chenyue Li | Xiaoxiao Sun | Yubo Deng | Qinlin Zhao | Serena Yeung-Levy | Yuhui Zhang
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
We introduce ScreenHaystack, a dynamic needle-in-a-haystack benchmark for evaluating spatial reliability in GUI grounding. Instead of testing each target at a fixed position, ScreenHaystack systematically relocates controlled target icons across high-resolution GUI backgrounds and measures whether models can localize them consistently. Using this benchmark, we find that leading GUI grounding models, including Qwen3-VL, UI-TARS, GTA, and UI-Venus, exhibit blind zones: spatial regions where grounding accuracy drops sharply despite fixed target appearance and instruction. These blind zones transfer to unseen ScreenSpot-Pro examples: targets inside blind zones are consistently harder to ground, with Qwen3-VL dropping by 16.1 percentage points, and controlled relocation shows that moving targets into blind zones decreases accuracy while moving them out improves accuracy. We further show through controlled synthetic experiments that uneven spatial coverage in training data can induce such blind zones. Therefore, we propose a simple strategy, blind-zone-oriented augmentation, which adds supervision in blind zones and improves ScreenSpot-Pro accuracy over both original and random augmented Click-100k fine-tuning.
Solve the Missing First Step: Can VLMs Standardize Raw Heterogeneous Medical Data?
Xin Chen | Dongliang Xu | Cunhao Zhu | Xudong Luo | Haoyang Lyu | Xiaoxiao Sun | Serena Yeung-Levy | Yue Yao
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Xin Chen | Dongliang Xu | Cunhao Zhu | Xudong Luo | Haoyang Lyu | Xiaoxiao Sun | Serena Yeung-Levy | Yue Yao
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
As vision-language models (VLMs) are increasingly applied to medical AI, existing benchmarks mainly focus on evaluating their diagnostic ability over given medical images and texts, implicitly assuming that standardized medical images, texts, or question–answer pairs are already prepared. However, this assumption does not hold when we apply VLMs in real clinical practice, where medical data is often raw, heterogeneous, and fragmented across different sources. In this paper, we study this missing step, i.e., raw medical data standardization. Specifically, models are given raw dataset folders and evaluated on their ability to identify source formats, convert raw medical images into VLM-compatible visual inputs, extract relevant textual information, and organize the results into structured image-text pairs. To construct this Medical Data Standardization Benchmark (MDS-Bench), we manually annotate 1,939 raw medical data standardization tasks covering diverse clinical practice, radiology modalities, annotation formats, and directory layouts. Extensive experiments show that even the best performing VLM, i.e., Gemini 3 Flash, achieves only a 48.6% end-to-end success rate. Our research highlights raw medical data standardization as a critical bottleneck for medical AI diagnosis in real practice.