Yunpeng Bao
Author directory2026
DeskCraft: Benchmarking Desktop Agents on Professional Workflows and Human-in-the-Loop Collaboration
Wenkai Wang | Tao Xiong | Jingchen Ni | Yunpeng Bao | Xiyun Li | Tianqi Liu | Hongcan Guo | Zilong Huang | Shengyu Zhang
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Wenkai Wang | Tao Xiong | Jingchen Ni | Yunpeng Bao | Xiyun Li | Tianqi Liu | Hongcan Guo | Zilong Huang | Shengyu Zhang
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Real-world professional desktop workflows in specialized creative and engineering software unfold over long horizons and often require human-in-the-loop coordination, where agents proactively seek necessary information and users provide additional instructions, clarifications, feedback, or corrections as the task progresses.Yet existing desktop GUI benchmarks mostly reduce this setting to short, simplified tasks with all user instructions provided upfront.To address this issue, we introduce **DeskCraft**, a desktop GUI benchmark targeting long horizon creative and engineering workflows and proactive human-agent collaboration. DeskCraft organizes tasks into a multilevel difficulty taxonomy, with long horizon tasks requiring over 50 execution steps, and covers professional creative software across design, video, audio, and 3D creation. Furthermore, DeskCraft formalizes human-agent collaboration into an interaction protocol covering *mid-turn* and *post-turn* exchanges. Mid-turn interaction captures both agent-initiated clarification under uncertainty and user-initiated interruption during execution, while post-turn interaction accommodates user-driven feedback after the agent signals completion, together spanning the full space of realistic collaboration patterns. We evaluate 18 proprietary and open source agents on 538 tasks and find that GPT-5.4 reaches **31.6%** on standard tasks and **27.6%** on interactive tasks. Further analyses reveal persistent failures in long horizon workflow delivery and proactive clarification. We will open-source all evaluation codes, tasks, and data.