Looks Right, Works Right: A Project-Level Benchmark for Multi-Screen Mobile App Generation

Fan Wu, Cuiyun Gao, Yiming Huang, Yang Xiao, Yujia Chen, Qing Liao


Abstract
Recent multimodal large language models can convert visual designs directly into executable code, but real mobile products require multiple screenshots to become a buildable codebase with shared components and working navigation. This project-level setting exposes three limits of existing design-to-code benchmarks: they focus on single-page generation rather than complete codebases, cannot evaluate cross-page navigation, and do not measure project-wide maintainability. We introduce MobileForge, the first benchmark for project-level multi-screen mobile app generation, comprising 29 real mobile apps, 309 human-reviewed screens, structured page-relationship annotations, and 701 navigation test specifications. MobileForge supports five-axis evaluation of build, navigation, visual fidelity, code maintainability, and efficiency. We also propose state-isolated navigation testing to avoid cascading failures in navigation evaluation and an anchor-referenced list-wise visual evaluation protocol to improve visual-judge reliability. Across 174 end-to-end runs on six frontier multimodal LLMs, current models can build mobile-app projects that compile and reach the correct pages, but interactive navigation remains unreliable and visual fidelity and maintainability still lag. The benchmark and supporting materials are available at https://anonymous.4open.science/r/MobileForge-E2E5/.
Anthology ID:
2026.emnlp-main.2650
Volume:
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Month:
October
Year:
2026
Address:
Budapest, Hungary
Editors:
Hung-Yi Lee, Sunipa Dev, Jessy Li, Desmond Elliott
Venue:
EMNLP
SIG:
SIGDAT
Publisher:
Association for Computational Linguistics
Note:
Pages:
60294–60319
Language:
URL:
https://aclanthology.org/2026.emnlp-main.2650/
DOI:
Bibkey:
Cite (ACL):
Fan Wu, Cuiyun Gao, Yiming Huang, Yang Xiao, Yujia Chen, and Qing Liao. 2026. Looks Right, Works Right: A Project-Level Benchmark for Multi-Screen Mobile App Generation. In Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing, pages 60294–60319, Budapest, Hungary. Association for Computational Linguistics.
Cite (Informal):
Looks Right, Works Right: A Project-Level Benchmark for Multi-Screen Mobile App Generation (Wu et al., EMNLP 2026)
Copy Citation:
PDF:
https://aclanthology.org/2026.emnlp-main.2650.pdf