Chaoyang Zhao

Author directory

2026

Recent multimodal large language models achieve promising results on visual reasoning tasks, approaching human-level performance on existing STEM benchmarks. However, these capabilities degrade on complex real-world scenes. We identify two intertwined causes: chain-of-thought reasoning remains largely confined to the linguistic space, and training data rarely requires each step to be anchored in visual evidence. In this work, we introduce Visual Augmentation Primitives, which leverages high-quality instruction-guided generation models to synthesize diverse photorealistic training samples and construct coherent multimodal reasoning chains throughout the generation process, requiring models to unify visual perception with logical reasoning at every step. We further introduce multimodal chain-of-thought rubric-based reinforcement learning, which extends rubric-based reward design from the text-only domain to visual reasoning, scoring intermediate reasoning steps for logical validity, completeness, and consistency, and construct the UNREAL benchmark to systematically evaluate visual reasoning on hyperrealistic scenes across controllable difficulty levels. Experiments show that a 4B model built on this framework surpasses 8B baselines, with the approach scaling consistently to 32B across visual reasoning and VQA tasks. All models, data, and code will be publicly released.

2025

Text-to-image synthesis has made remarkable progress, yet accurately interpreting complex and lengthy prompts remains challenging, often resulting in semantic inconsistencies and missing details. Existing solutions, such as fine-tuning, are model-specific and require training, while prior automatic prompt optimization (APO) approaches typically lack systematic error analysis and refinement strategies, resulting in limited reliability and effectiveness. Meanwhile, test-time scaling methods operate on fixed prompts and on noise or sample numbers, limiting their interpretability and adaptability. To solve these, we introduce a flexible and efficient test-time prompt optimization strategy that operates directly on the input text. We propose a plug-and-play multi-agent system called GenPilot, integrating error analysis, clustering-based adaptive exploration, fine-grained verification, and a memory module for iterative optimization. Our approach is model-agnostic, interpretable, and well-suited for handling long and complex prompts. Simultaneously, we summarize the common patterns of errors and the refinement strategy, offering more experience and encouraging further exploration. Experiments on DPG-bench and Geneval with improvements of up to 16.9% and 5.7% demonstrate the strong capability of our methods in enhancing the text and image consistency and structural coherence of generated images, revealing the effectiveness of our test-time prompt optimization strategy. The code is available at https://github.com/27yw/GenPilot.