Wenjie Ai
Author directory2026
OS-Marathon: Benchmarking Computer-Use Agents on Vast-Horizon, Repetitive Tasks
Jing Wu | Wenjie Ai | Daphne Barretto | Yiye Chen | Qingyu Chen | Yuhang He | Pranit Chawla | Nicholas Gydé | Yanan Jian | Vibhav Vineet
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Jing Wu | Wenjie Ai | Daphne Barretto | Yiye Chen | Qingyu Chen | Yuhang He | Pranit Chawla | Nicholas Gydé | Yanan Jian | Vibhav Vineet
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Vast-horizon, repetitive workflows are common in daily routines, e.g., processing expense reports from a stack of receipts, organising a collection of PDF annotations into structured notes, and are tedious for humans, with execution length scaling with the volume of data to process. They are, however, well-suited to autonomous agents, since their structured, recurring sub-workflows follow a logic that can be systematically learned. Yet no existing benchmark evaluates agents specifically on vast-horizon repetitive capabilities, leaving the problem largely underexplored. To bridge this gap, we establish OS-Marathon, comprising 100 vast-horizon repetitive tasks across 5 scenarios and 10 domains, on which our evaluation reveals that leading state-of-the-art (SOTA) CUAs struggle substantially. We further find that introducing a task orchestrator to decompose the workflow into per-instance subtasks fails to mitigate the difficulty: errors accumulate and propagate across solver agents, indicating that naive decomposition is insufficient for these tasks. We then explore a cost-effective personalisation strategy, i.e. GraphDemo, that adapts general agents to such tasks from a single human demonstration of the recurring sub-workflow logic. Extensive experiments show both the real-world challenge of vast-horizon repetitive tasks and the improvement provided by human demonstration in this setting.