Kartik Mathur

Author directory

2026

We introduce GUI-360°, a large-scale dataset and benchmark suite for desktop productivity computer-using agents (CUAs). GUI-360° provides an automated pipeline for query sourcing, template-based environment construction, task instantiation, batched execution, and LLM-assisted quality filtering in Windows office applications. The corpus contains 131K validated successful action steps across 17K trajectories for training and evaluation, together with 1.09M failed execution steps for failure analysis and future recovery training. Each step provides screenshots, accessibility metadata, instantiated goals, reasoning traces, and structured GUI/API actions. GUI-360° supports GUI grounding, screen parsing, and action prediction under a hybrid GUI+API action space. Experiments show that current vision–language models struggle with fine-grained grounding and stepwise action prediction, while supervised fine-tuning on GUI-360° brings consistent in-domain and external benchmark gains. Our code and dataset are available at https://github.com/2020-qqtcg/GUI-360 and https://huggingface.co/datasets/vyokky/GUI-360.