Dayong Yang
Author directory2026
MACBench: Benchmarking Multi-turn Agentic Coding with Evolving Requirements in Frontend Web Development
Jiayi Gui | Bosi Wen | Haoke Zhang | Pei Ke | Yukuo Cen | Dayong Yang | Minlie Huang | Jie Tang | Cunxiang Wang
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Jiayi Gui | Bosi Wen | Haoke Zhang | Pei Ke | Yukuo Cen | Dayong Yang | Minlie Huang | Jie Tang | Cunxiang Wang
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Modern coding agents are increasingly deployed for realistic software engineering, yet existing benchmarks primarily evaluate static, fixed-target tasks, failing to capture the long-horizon, multi-turn nature of real-world development. We introduce MACBench, a novel benchmark evaluating agents on sustained frontend web application development where text-based user requirements continuously evolve within a persistent workspace. Comprising 28 repository instances across 8 domains, MACBench features 699 dialogue turns and 956 evaluation items. It explicitly models realistic interaction patterns, challenging agents to maintain cross-turn coherence and continuous executability. Our dual-branch evaluation framework assesses both process robustness and final artifact quality, uniquely integrating intermediate viability checks with bounded runtime-feedback repair. Benchmarking 4 contemporary models reveals a severe reliability gap: the best-performing agent achieves only a 70.6% checklist success rate and a 10.7% strict instance success rate, struggling to satisfy requirements accumulated over long sessions. Furthermore, ablation studies reveal diminishing and sometimes negative returns from runtime-feedback repair: while it improves checklist success by an average of 10.4% and aids immediate failure recovery, excessive repair attempts frequently degrade final functional outcomes, suggesting they may introduce regressions or side effects in the evolving codebase. Finally, failure analysis identifies complex cross-turn application logic as the dominant bottleneck, accounting for 33–50% of unresolved errors.