Ziwen Wang

Author directory

2026

GUI agents powered by Multimodal Large Language Models (MLLMs) are advancing rapidly, yet most existing benchmarks evaluate only task-level success or failure—leaving a fundamental question unanswered: what capabilities do GUI agents most need to succeed in real-world settings? We introduce UniGUI-Bench, a benchmark that jointly verifies task-level outcomes and process-level capability diagnosis. It spans 6 device platforms and 23 application scenarios with bilingual support, comprising over 10,000 evaluation items. Its core design links the two evaluation axes: each task failure can be traced to specific capability deficiencies, enabling fine-grained attribution beyond binary pass/fail. Experiments on 14 frontier models reveal three findings: (1) model rankings diverge substantially from single-platform leaderboards under broader evaluation; (2) different scenarios impose distinct capability bottlenecks identifiable through our attribution framework; (3) targeted training on diagnosed bottleneck dimensions yields consistent improvements on both UniGUI-Bench and OSWorld, providing preliminary evidence toward closing the evaluation–improvement loop. The UniGUI-Bench dataset and evaluation code are available at https://huggingface.co/datasets/AGI-Eval/UniGUI-Bench.
Search Agents (SAs) typically leverage large language models (LLMs) to support complex information-seeking tasks by autonomously exploring web sources and synthesizing information into comprehensive responses. For SAs evaluation, prior benchmarks mainly focus on specialized tasks that are unlikely to arise in real-world user scenarios. Moreover, their reliance on coarse task-level rubrics often limits evaluation interpretability. To bridge this gap, we introduce DailyReport, an open-ended benchmark to evaluate SA capabilities on daily search tasks. It contains 150 open-ended tasks with 3,546 associated rubrics, capturing widely discussed and timely information demands of real-world users. Each task is decomposed into subtasks and evaluated with cascade rubrics across disentangled dimensions. Through cascade performance attribution and user-centric aggregation, we derive highly interpretable scores for each dimension, along with a user preference score. Our results on 17 agentic systems show that current systems still fall short of users’ expectations. DailyReport will be open-sourced to facilitate future research.
We present AMO-Bench, an Advanced Mathematical reasoning benchmark with Olympiad level or even higher difficulty, comprising 50 human-crafted problems. Existing benchmarks have widely leveraged high school math competitions for evaluating mathematical reasoning capabilities of large language models (LLMs). However, many existing math competitions are becoming less effective for assessing top-tier LLMs due to performance saturation (e.g., AIME24/25). To address this, AMO-Bench introduces more rigorous challenges by ensuring all 50 problems are (1) cross-validated by experts to meet at least the International Mathematical Olympiad (IMO) difficulty standards, and (2) entirely original problems to prevent potential performance leakages from data memorization. Experimental results across 36 LLMs on AMO-Bench highlights three key findings: (1) high-level mathematical reasoning remains challenging for current LLMs, with even the best-performing model achieving only 63.1% accuracy and most LLMs scoring below 50%; (2) scaling test-time compute remains a highly effective strategy for substantially improving reasoning performances, and (3) open-source models are progressively narrowing the performance gap with proprietary models. Additionally, we conduct further analysis about reasoning efficiency, volatility, and cross-lingual robustness, providing deeper insights behind the reasoning performances.