Ying-Cong Chen
Author directory2026
UniData: Universal Multimodal Instruction Generation Pipeline
Jiaqi Tang | Yi-Feng Wu | Yuting Zhang | Hao LU | Bowen Fu | Qing-Guo Chen | Xiaogang Xu | Yuwei Hu | Shiyin Lu | Wei Wei | Lei Zhang | Zhao Xu | Weihua Luo | Qifeng Chen | Ying-Cong Chen
Findings of the Association for Computational Linguistics: EMNLP 2026
Jiaqi Tang | Yi-Feng Wu | Yuting Zhang | Hao LU | Bowen Fu | Qing-Guo Chen | Xiaogang Xu | Yuwei Hu | Shiyin Lu | Wei Wei | Lei Zhang | Zhao Xu | Weihua Luo | Qifeng Chen | Ying-Cong Chen
Findings of the Association for Computational Linguistics: EMNLP 2026
Multimodal Large Language Models (MLLMs) are increasingly being applied in a wider range of real-world scenarios. However, due to the substantial labor cost, creating high-quality multimodal instruction datasets for MLLMs remains a significant challenge. Although some methods propose to generate instruction data, they often face limitations in modality support and struggle with generating multi-round instructions. To address these problems, we introduce UniData, a universal instruction generation pipeline, to transform simple user requirements into multi-round, multimodal instructions. Specifically, UniData first expands user requirements into multiple diverse events. Using these events, UniData then integrates an any-to-any large model for multimodal instruction generation. Finally, UniData enhances data quality by correcting irrelevant and redundant inference flow, leveraging correlations between instruction rounds. To train this pipeline, we also build UniDataset, a dataset comprising 20,000 entries across nine modalities for improved multimodal generation. Our experiments demonstrate that UniData achieves SOTA performance in data quality and can also enhance the understanding and generation capabilities of other multimodal models.
SceneLM: 3D-Aware Language Models for Editable 3D Scene Synthesis
Xingbo Yao | Xiaoyu Chen | Doudou Zhang | Mingzhi Sheng | Boyuan Cao | Ying-Cong Chen | Hui Xiong
Findings of the Association for Computational Linguistics: ACL 2026
Xingbo Yao | Xiaoyu Chen | Doudou Zhang | Mingzhi Sheng | Boyuan Cao | Ying-Cong Chen | Hui Xiong
Findings of the Association for Computational Linguistics: ACL 2026
Synthesizing an editable 3D scene from a single RGB image is central to content creation, embodied-agent data generation, and AR/VR, yet remains challenging to achieve both high-fidelity reconstruction and convenient interactive editing. Existing geometry-based pipelines produce high-quality 3D results but are typically hard to refine without rerunning the full process, while LLM-driven procedural systems enable interactive tool use but are mostly text-driven and lack precise metric 3D understanding from images. We present SceneLM, a language-model-based framework that grounds 3D scene synthesis in visual evidence by recovering an executable metric 3D layout directly from a single image. Given an RGB image (and camera intrinsics when available), SceneLM outputs a JSON-form layout specifying each object’s category, 3D center, size, and discretized yaw, and then deterministically executes this layout with a tool suite to instantiate, place, and edit objects for iterative refinement. To train metric layout recovery at scale, we curate five datasets covering diverse indoor, outdoor, and tabletop scenes and convert heterogeneous 3D annotations into a unified instruction-tuning format. To improve numerical stability and metric accuracy while preserving the text interface, we augment autoregressive JSON generation with a lightweight geometry prediction branch and dual supervision. Experiments show that SceneLM substantially improves single-image 3D layout estimation over strong open and proprietary MLLM baselines, and yields higher-quality end-to-end scene generation in geometric consistency, physical plausibility, semantic alignment, and realism.
Streaming Communication in Multi-Agent Reasoning
Zhen Yang | Xiaogang Xu | Wen Wang | Cong Chen | Xander Xu | Ying-Cong Chen
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Zhen Yang | Xiaogang Xu | Wen Wang | Cong Chen | Xander Xu | Ying-Cong Chen
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Multi-agent reasoning systems adopt a generate-then-transfer paradigm that forces end-to-end latency to scale linearly with pipeline depth. We introduce StreamMA, a multi-agent reasoning system that streams each reasoning step to downstream agents as soon as it is generated, pipelining adjacent agents and thus reducing latency. Surprisingly, this pipelining also improves effectiveness: because multi-step reasoning quality is non-uniform and early steps are more reliable than later ones, working with these reliable early steps instead of the full chain prevents error-prone late steps from misleading downstream agents. We formalize both advantages with the first closed-form joint analysis of stream, serial, and single protocols, deriving the effectiveness ordering, speedup upper bound, and cost ratio. Across eight reasoning benchmarks spanning mathematics, science, and code, three frontier LLMs (Claude Opus 4.6, GPT-5.4, and GLM-5.2), and three topologies (Chain, Tree, Graph), StreamMA outperforms both baselines (avg.+7.3pp, max +22.4pp on HMMT2026; Claude Opus 4.6-High). Beyond these contributions, we discover a step-level scaling law: increasing per-agent steps consistently improves both effectiveness and efficiency, a new scaling dimension orthogonal to and composable with agent-count scaling.
Policy and World Modeling Co-Training for Language Agents
Ning Lu | Baijiong Lin | Shengcai Liu | Jiahao Wu | Haoze Lv | Yanbin Wei | Lingting Zhu | Shengju Qian | Xin Wang | Ying-Cong Chen | Qi Wang | Ke Tang
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Ning Lu | Baijiong Lin | Shengcai Liu | Jiahao Wu | Haoze Lv | Yanbin Wei | Lingting Zhu | Shengju Qian | Xin Wang | Ying-Cong Chen | Qi Wang | Ke Tang
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Reinforcement learning (RL) improves large language model (LLM) agents by teaching them which actions lead to high rewards, but provides little supervision on what those actions do to the environment. World modeling (WM) can fill this gap, yet existing approaches often require separate simulators, extra training stages, or additional inference-time computation. We observe that on-policy RL rollouts already contain the needed signal: each transition pairs an action with its resulting next observation. Based on this observation, we propose PaW, a Policy and World modeling co-training framework that adds auxiliary WM supervision to the same policy during RL, without changing the inference paradigm. To make auxiliary WM supervision informative and stable, PaW introduces three components: action-entropy-based WM data selection, noise-tolerant WM loss, and reward-adaptive loss balancing. Experiments on three agentic task benchmarks show consistent improvements over strong RL baselines across models and RL algorithms. These results suggest that standard RL rollouts are a practical source of WM supervision for language-agent training.
WorldLines: Benchmarking and Modeling Long-Horizon Stateful Embodied Agents
Yehang Zhang | Jianchong Su | Haojian Huang | Yifan Chang | Tianhao Zhou | Xinli Xu | Yingjie Xu | Yinchuan Li | Zexi Li | Ying-Cong Chen
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Yehang Zhang | Jianchong Su | Haojian Huang | Yifan Chang | Tianhao Zhou | Xinli Xu | Yingjie Xu | Yinchuan Li | Zexi Li | Ying-Cong Chen
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
To assist humans over extended periods in real homes, embodied agents must remember user routines, world states, and past interactions. Existing long-term memory benchmarks mainly evaluate language-centric retrieval and question answering, while embodied benchmarks often focus on short-horizon task execution without testing long-term memory use in dynamic environments. We introduce WorldLines, a project-driven benchmark for long-horizon embodied household assistance. It constructs temporally extended household traces with dialogues, actions, execution feedback, object and device state changes, and converts them into evidence-linked samples for Memory QA and Embodied Task Planning. We further propose ObsMem, an observer-grounded memory framework that maintains visibility-aware memories and action-native state trails for state-aware decisions. Experiments reveal persistent challenges in partial observability, overwritten world states, and translating long-term memory into embodied plans, while ObsMem offers a stronger reference architecture for this setting.
Less is More: Improving LLM Reasoning with Minimal Test-Time Intervention
Zhen Yang | Mingyang Zhang | Feng Chen | Ganggui Ding | Liang Hou | Xin Tao | Ying-Cong Chen
Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
Zhen Yang | Mingyang Zhang | Feng Chen | Ganggui Ding | Liang Hou | Xin Tao | Ying-Cong Chen
Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
Recent progress in large language models (LLMs) has focused on test-time scaling to improve reasoning via increased inference computation, but often at the cost of efficiency. We revisit test-time behavior and uncover a simple yet underexplored phenomenon: reasoning uncertainty is highly localized—only a small subset of high-entropy tokens dominantly affects output correctness. Motivated by this, we propose Minimal Test-Time Intervention (MTI), a training-free framework that enhances reasoning accuracy and stability with minimal overhead. MTI includes: (i) Selective CFG intervention, applying classifier-free guidance only at uncertain positions; and (ii) Lightweight negative-prompt guidance, reusing the main model’s KV cache to approximate unconditional decoding efficiently. MTI yields consistent gains across general, coding, and STEM tasks—e.g., +9.28% average improvement on six benchmarks for DeepSeek-R1-7B and +11.25% on AIME2024 using Ling-mini-2.0—while remaining highly efficient.
2025
PreGenie: An Agentic Framework for High-quality Visual Presentation Generation
Xiaojie Xu | Xinli Xu | Sirui Chen | Haoyu Chen | Fan Zhang | Ying-Cong Chen
Findings of the Association for Computational Linguistics: EMNLP 2025
Xiaojie Xu | Xinli Xu | Sirui Chen | Haoyu Chen | Fan Zhang | Ying-Cong Chen
Findings of the Association for Computational Linguistics: EMNLP 2025
Visual presentations are vital for effective communication. Early attempts to automate their creation using deep learning often faced issues such as poorly organized layouts, inaccurate text summarization, and a lack of image understanding, leading to mismatched visuals and text. These limitations restrict their application in formal contexts like business and scientific research. To address these challenges, we propose PreGenie, an agentic and modular framework powered by multimodal large language models (MLLMs) for generating high-quality visual presentations.PreGenie is built on the Slidev presentation framework, where slides are rendered from Markdown code. It operates in two stages: (1) Analysis and Initial Generation, which summarizes multimodal input and generates initial code, and (2) Review and Re-generation, which iteratively reviews intermediate code and rendered slides to produce final, high-quality presentations. Each stage leverages multiple MLLMs that collaborate and share information. Comprehensive experiments demonstrate that PreGenie excels in multimodal understanding, outperforming existing models in both aesthetics and content consistency, while aligning more closely with human design preferences.
Orchestrating Audio: Multi-Agent Framework for Long-Video Audio Synthesis
Yehang Zhang | Xinli Xu | Xiaojie Xu | Doudou Zhang | Li Liu | Ying-Cong Chen
Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing
Yehang Zhang | Xinli Xu | Xiaojie Xu | Doudou Zhang | Li Liu | Ying-Cong Chen
Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing
Video-to-audio synthesis, which generates synchronized audio for visual content, critically enhances viewer immersion and narrative coherence in film and interactive media. However, video-to-audio dubbing for long-form content remains an unsolved challenge due to dynamic semantic shifts, audio diversity and the absence of dedicated datasets. While existing methods excel in short videos, they falter in long scenarios (e.g., movies) due to fragmented synthesis and inadequate cross-scene consistency. We propose LVAS-Agent, a multi-agent framework that offers a coordinated, multi-component approach to long-video audio generation. Our approach decomposes long-video synthesis into four steps including scene segmentation, script generation, audio design and audio synthesis. To enable systematic evaluation, we introduce LVAS-Bench, the first benchmark with 207 professionally curated long videos spanning diverse scenarios. Experiments show that our method outperforms state-of-the-art V2A models in overall audio synthesis quality.
Search
Fix author
Co-authors
- Xinli Xu 3
- Xiaogang Xu 2
- Xiaojie Xu 2
- Zhen Yang 2
- Doudou Zhang 2
- Yehang Zhang 2
- Boyuan Cao 1
- Yifan Chang 1
- Cong Chen (陈聪) 1
- Feng Chen 1
- Haoyu Chen 1
- Qifeng Chen 1
- Qing-Guo Chen 1
- Sirui Chen 1
- Xiaoyu Chen 1
- Ganggui Ding 1
- Bowen Fu 1
- Liang Hou 1
- Yuwei Hu 1
- Haojian Huang 1
- Hao LU 1
- Yinchuan Li 1
- Zexi Li 1
- Baijiong Lin 1
- Li Liu 1
- Shengcai Liu 1
- Ning Lu 1
- Shiyin Lu 1
- Weihua Luo 1
- Haoze Lv 1
- Shengju Qian 1
- Mingzhi Sheng 1
- Jianchong Su 1
- Jiaqi Tang 1
- Ke Tang 1
- Xin Tao 1
- Qi Wang 1
- Wen Wang 1
- Xin Wang 1
- Wei Wei 1
- Yanbin Wei 1
- Jiahao Wu 1
- Yi-Feng Wu 1
- Hui Xiong 1
- Xander Xu 1
- Yingjie Xu 1
- Zhao Xu 1
- Xingbo Yao 1
- Fan Zhang 1
- Lei Zhang 1
- Mingyang Zhang 1
- Yuting Zhang 1
- Tianhao Zhou 1
- Lingting Zhu 1