Bowen Fu
Author directoryOther people with similar names: Bowen Fu
2026
MacAgentBench: Benchmarking AI Agents on Real-World macOS Desktop
Yikun Fu | Bowen Fu | Zhenyu Wu | Shuang Cheng | Xiaowei Sun | Bowen Yang | Zehao Li | Yibo Zhao | Zichen Ding | Zhoumianze Liu | Shijie Wang | Biqing Qi | Bowen Zhou
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Yikun Fu | Bowen Fu | Zhenyu Wu | Shuang Cheng | Xiaowei Sun | Bowen Yang | Zehao Li | Yibo Zhao | Zichen Ding | Zhoumianze Liu | Shijie Wang | Biqing Qi | Bowen Zhou
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Computer use agents (CUAs) have advanced rapidly in desktop automation, and a growing number of users deploy CUAs such as OpenClaw on Mac Mini for always-on automation. However, existing benchmarks, including those for macOS, evaluate agents without framework augmentation and rely on binary evaluation. As a result, they fail to capture both the framework capabilities leveraged by modern CUAs and the partial progress on long-horizon, multi-application tasks. We present MacAgentBench, a comprehensive macOS agent benchmark comprising 169 seed tasks expanded into 676 task instances across 25 applications, with nearly 60% involving both GUI and CLI interaction. The benchmark adopts deterministic rule-based evaluation and introduces fine-grained multi-checkpoint scoring with capability annotations for multi-application tasks. Experiments across three frameworks and 16 models show that the best configuration, Claude Opus 4.6 on OpenClaw, attains 73.7% Pass@1, while this advantage is primarily driven by the skill library rather than by framework design. Fine-grained metrics further reveal that models with similar Pass@1 can differ substantially in sub-goal completion.