Yinghao Wu
Author directory2026
VEN-VL: A Visual Ensemble MoE Framework for Effective and Efficient Multi-Modal Understanding
Yinghao Wu | Zhuoyan Luo | Yiyao Yu | Zhaojian Yu | Yujiu Yang | Xiao-Ping Zhang
Findings of the Association for Computational Linguistics: EMNLP 2026
Yinghao Wu | Zhuoyan Luo | Yiyao Yu | Zhaojian Yu | Yujiu Yang | Xiao-Ping Zhang
Findings of the Association for Computational Linguistics: EMNLP 2026
Despite the remarkable progress achieved by recent efficient methods in accelerating multimodal understanding, they still suffer from noticeable performance degradation. Their emphasis on the high compression ratio of a single visual clue and reliance on the heuristic pruning strategy with coarse attention alignment incurs a bottleneck on the information *capacity* and *density* of visual tokens. Addressing this limitation, we propose VEN-VL, a visual ensemble MoE framework for effective and efficient perception following the *enrich then compact* principle. Specifically, we first enrich the information capacity by unifying the visual representations of different perspectives, and then progressively compact it with adaptive routers in specialized visual experts to enhance the information density. Furthermore, we incorporate the reconstruction ability of vanilla structure via explicit visual supervision, facilitating crucial information preservation. Experimental results demonstrate our superiority in complex visual tasks with few information-condensed tokens, which effectively bridges the gap between performance and efficiency. Code will be available.
2025
Z1: Efficient Test-time Scaling with Code
Zhaojian Yu | Yinghao Wu | Yilun Zhao | Arman Cohan | Xiao-Ping Zhang
Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing: Industry Track
Zhaojian Yu | Yinghao Wu | Yilun Zhao | Arman Cohan | Xiao-Ping Zhang
Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing: Industry Track
Large Language Models (LLMs) can achieve enhanced complex problem-solving through test-time computing scaling, yet this often entails longer contexts and numerous reasoning token costs. In this paper, we propose an efficient test-time scaling method that trains LLMs on code-related reasoning trajectories, facilitating their reduction of excess thinking tokens while maintaining performance.First, we create Z1-Code-Reasoning-107K, a curated dataset of simple and complex coding problems paired with their short and long solution trajectories. Second, we present a novel Shifted Thinking Window to mitigate overthinking overhead by removing context-delimiting tags (e.g., <think>...</think>) and capping reasoning tokens. Trained with long and short trajectory data and equipped with Shifted Thinking Window, our model, Z1-7B, demonstrates the ability to adjust its reasoning level as the complexity of problems and exhibits efficient test-time scaling across different reasoning tasks that matches R1-Distill-Qwen-7B performance with about 30% of its average thinking tokens.Notably, fine-tuned with only code trajectories, Z1-7B demonstrates generalization to broader reasoning tasks (47.5% on GPQA Diamond). Our analysis of efficient reasoning elicitation also provides valuable insights for future research.