YiMing Cheng
Author directoryAlso published as: Yiming Cheng
Other people with similar names: Yiming Cheng
2026
HiSVR: Hierarchical Semantic Visual Reconstruction for Unified Multimodal Models
Tailai Chen | Weining Wang | Yuchen Liu | Jun Fu | Jie Jiang | Yiming Cheng | Jing Liu
Findings of the Association for Computational Linguistics: EMNLP 2026
Tailai Chen | Weining Wang | Yuchen Liu | Jun Fu | Jie Jiang | Yiming Cheng | Jing Liu
Findings of the Association for Computational Linguistics: EMNLP 2026
Unified multimodal models use a shared Transformer backbone for visual understanding and image generation. Existing post-training signals primarily constrain final outputs, output-side representations, or task-level behavior, providing limited supervision for how visual semantics are organized across intermediate layers. We propose HiSVR, a hierarchical semantic visual reconstruction method for post-training unified multimodal models. A frozen semantic visual tokenizer extracts residual semantic codes from each training image, which supervise image-associated hidden states at different Transformer depths. Shallow layers predict shorter semantic-code prefixes, while deeper layers reconstruct longer prefixes, yielding coarse-to-fine supervision over intermediate representations. To balance the differing optimization difficulty across depths and prefix lengths, we introduce an uncertainty-weighted progressive reconstruction loss. HiSVR is used only during training and removed at inference, preserving the original architecture and inference cost. Experiments on unified multimodal backbones show that HiSVR improves visual understanding while maintaining image generation quality. These results indicate that hierarchical semantic reconstruction effectively regularizes intermediate representations in unified multimodal models.
MTAVG-Bench: A Diagnostic Benchmark for Multi-Talker Dialogue-Centric Audio-Video Generation
Yanghao Zhou | Haitian Li | Rexar Lin | Heyan Huang | Jinxing Zhou | Changsen Yuan | Tian Lan | Ziqin Zhou | Yudong Li | Jiajun Xu | Jingyun Liao | YiMing Cheng | Xuefeng Chen | Xian-Ling Mao | Yousheng Feng
Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
Yanghao Zhou | Haitian Li | Rexar Lin | Heyan Huang | Jinxing Zhou | Changsen Yuan | Tian Lan | Ziqin Zhou | Yudong Li | Jiajun Xu | Jingyun Liao | YiMing Cheng | Xuefeng Chen | Xian-Ling Mao | Yousheng Feng
Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
Recent advances in text-to-audio-video (T2AV) generation have enabled models to synthesize audio-visual videos with multi-participant dialogues. However, existing evaluation benchmarks remain largely designed for human-recorded videos or single-speaker settings. As a result, structural failures in generated multi-talker dialogue videos, such as identity drift, unnatural turn transitions, and audio-visual misalignment, cannot be effectively diagnosed. To address this issue, we introduce MTAVG-Bench, a failure-driven diagnostic benchmark for multi-talker dialogue-centric audio-video generation. MTAVG-Bench is built via a semi-automatic pipeline, where 1.8k videos are generated using mainstream T2AV models with carefully designed prompts, yielding 2.4k manually annotated QA pairs for fine-grained failure diagnosis. The benchmark evaluates multi-speaker dialogue generation at four levels: audio-visual signal fidelity, temporal attribute consistency, social interaction, and cinematic expression. Built on a hierarchical failure taxonomy and a targeted QA protocol, MTAVG-Bench is primarily designed to evaluate whether proprietary and open-source omni-models can reliably identify failure modes in multi-speaker T2AV outputs. We benchmark 12 proprietary and open-source omni-models on MTAVG-Bench, with Gemini 3 Pro achieving the strongest overall performance, while leading open-source models remain competitive in signal fidelity and consistency. Overall, MTAVG-Bench enables fine-grained failure analysis for rigorous model comparison and targeted video generation refinement.