Liya Ma
Author directory2026
UniGUI-Bench: A Real-World Diagnostic Benchmark — What Capabilities Do GUI Agents Most Need?
Jiaxing Liu | Ruoqi Sun | Liya Ma | FengJiao Chen | Ziwen Wang | Xuezhi Cao
Findings of the Association for Computational Linguistics: EMNLP 2026
Jiaxing Liu | Ruoqi Sun | Liya Ma | FengJiao Chen | Ziwen Wang | Xuezhi Cao
Findings of the Association for Computational Linguistics: EMNLP 2026
GUI agents powered by Multimodal Large Language Models (MLLMs) are advancing rapidly, yet most existing benchmarks evaluate only task-level success or failure—leaving a fundamental question unanswered: what capabilities do GUI agents most need to succeed in real-world settings? We introduce UniGUI-Bench, a benchmark that jointly verifies task-level outcomes and process-level capability diagnosis. It spans 6 device platforms and 23 application scenarios with bilingual support, comprising over 10,000 evaluation items. Its core design links the two evaluation axes: each task failure can be traced to specific capability deficiencies, enabling fine-grained attribution beyond binary pass/fail. Experiments on 14 frontier models reveal three findings: (1) model rankings diverge substantially from single-platform leaderboards under broader evaluation; (2) different scenarios impose distinct capability bottlenecks identifiable through our attribution framework; (3) targeted training on diagnosed bottleneck dimensions yields consistent improvements on both UniGUI-Bench and OSWorld, providing preliminary evidence toward closing the evaluation–improvement loop. The UniGUI-Bench dataset and evaluation code are available at https://huggingface.co/datasets/AGI-Eval/UniGUI-Bench.