Liming Zheng
Author directory2026
MobileDreamer: Generative Sketch World Model for GUI Agent
Yilin Cao | Yufeng Zhong | Zhixiong Zeng | Liming Zheng | Jing Huang | Haibo Qiu | Peng Shi | Wenji Mao
Findings of the Association for Computational Linguistics: EMNLP 2026
Yilin Cao | Yufeng Zhong | Zhixiong Zeng | Liming Zheng | Jing Huang | Haibo Qiu | Peng Shi | Wenji Mao
Findings of the Association for Computational Linguistics: EMNLP 2026
Mobile GUI agents have shown strong potential in real-world automation and practical applications. However, most existing agents remain reactive, making decisions mainly from the current screen, which limits their performance on long-horizon tasks. To address this limitation, a natural approach is to use the forecasting ability of world models to support better decision-making. This is challenging because existing world models often lack spatial awareness when predicting post-action states and are too expensive for practical deployment. In this paper, we propose MobileDreamer, an efficient world-model-based lookahead framework to equip the GUI agents based on the future imagination provided by the world model. It consists of textual sketch world model and rollout imagination for GUI agent. Textual sketch world model forecasts post-action states through a learning process to transform digital images into key task-related sketches, and designs a novel order-invariant learning strategy to preserve the spatial information of GUI elements. The rollout imagination strategy for the GUI agent optimizes the action-selection process by leveraging the prediction capability of world model. Empirical results on Android World demonstrate that MobileDreamer consistently improves the task success rate across multiple base models, achieving an average performance gain of 5.3%. World model evaluations further verify that our textual sketch modeling accurately forecasts key GUI elements.
VinciCoder: Unifying Multimodal Code Generation via Coarse-to-fine Visual Reinforcement Learning
Xuanle Zhao | Deyang Jiang | Zhixiong Zeng | Lei Chen | Haoyue Yang | Haibo Qiu | Jing Huang | Yufeng Zhong | Liming Zheng | Yilin Cao | Lin Ma
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Xuanle Zhao | Deyang Jiang | Zhixiong Zeng | Lei Chen | Haoyue Yang | Haibo Qiu | Jing Huang | Yufeng Zhong | Liming Zheng | Yilin Cao | Lin Ma
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
While recent specialized multimodal code generation models excel in tasks like chart-to-code generation, their reliance on single-task training limits generalization and hinders the development of VIsioN Code Intelligence. In this work, we introduce VinciCoder, a unified framework designed for generalized multimodal code generation. We first curate a large-scale SFT corpus comprising 1.3M direct generation pairs and 300k visual-based refinement tasks. This composition fosters self-refinement capabilities, enabling the model to directly rectify code to align with input images. Subsequently, we propose coarse-to-fine Visual Reinforcement Learning (ViRL) to overcome the brittleness of textual metrics in handling semantically equivalent but syntactically diverse code. By quantifying visual similarity across multi-scale patches, ViRL provides an implementation-agnostic reward mechanism that ensures high-fidelity alignment between rendered outputs and input visuals. Extensive experimental results across diverse benchmarks demonstrate that VinciCoder achieves superior performance, while comprehensive ablation studies validate the effectiveness of our proposed ViRL strategy. The data, code and model are available at https://github.com/DocTron-hub/VinciCoder.