Yu Zeng
Author directory2026
Internalizing Meta-Experience into Memory for Guided Reinforcement Learning in Large Language Models
Shiting Huang | Zecheng Li | Yu Zeng | Qingnan Ren | Zhen Fang | Qisheng Su | Kou Shi | Ziao Zhang | Lin Chen | Zehui Chen | Feng Zhao
Findings of the Association for Computational Linguistics: EMNLP 2026
Shiting Huang | Zecheng Li | Yu Zeng | Qingnan Ren | Zhen Fang | Qisheng Su | Kou Shi | Ziao Zhang | Lin Chen | Zehui Chen | Feng Zhao
Findings of the Association for Computational Linguistics: EMNLP 2026
Reinforcement Learning with Verifiable Rewards (RLVR) has emerged as an effective approach for enhancing the reasoning capabilities of Large Language Models (LLMs). Despite its efficacy, RLVR faces a meta-learning bottleneck: it lacks mechanisms for error attribution and experience internalization intrinsic to the human learning cycle beyond practice and verification, thereby limiting fine-grained credit assignment and reusable knowledge formation. We term such reusable knowledge representations derived from past errors as meta-experience. Based on this insight, we propose **M**eta-**E**xperience **L**earning (**MEL**), a novel framework that incorporates self-distilled meta-experience into the model’s parametric memory. Building upon standard RLVR, we introduce an additional design that leverages the LLM’s self-verification capability to conduct contrastive analysis on paired correct and incorrect trajectories, identify the precise bifurcation points where reasoning errors arise, and summarize them into generalizable meta-experience. The meta-experience is further internalized into the LLM’s parametric memory by minimizing the negative log-likelihood, which induces a language-modeled reward signal that bridges correct and incorrect reasoning trajectories and facilitates effective knowledge reuse. Experimental results demonstrate that MEL achieves consistent improvements on benchmarks, yielding 3.92%–4.73% Pass@1 gains across varying model sizes.
Video-DeepResearch: Towards the Next-Generation Multimodal Deepresearch Agent
Zhen Fang | Yu Zeng | Wenxuan Huang | Yiming Zhao | Shiting Huang | Tianfei Ren | Qi Lu | Qingnan Ren | Qisheng Su | Lionel Z. Wang | Qingyu Yin | Shuang Chen | Zehui Chen | Lin Chen | Zhenfei Yin | Yao Hu | Shaohui Lin | Wanli Ouyang | Shaosheng Cao | Feng Zhao
Findings of the Association for Computational Linguistics: EMNLP 2026
Zhen Fang | Yu Zeng | Wenxuan Huang | Yiming Zhao | Shiting Huang | Tianfei Ren | Qi Lu | Qingnan Ren | Qisheng Su | Lionel Z. Wang | Qingyu Yin | Shuang Chen | Zehui Chen | Lin Chen | Zhenfei Yin | Yao Hu | Shaohui Lin | Wanli Ouyang | Shaosheng Cao | Feng Zhao
Findings of the Association for Computational Linguistics: EMNLP 2026
We introduce Video-DeepResearch (Video-DR), extending multimodal agents from static images to continuous video streams, a setting that demands dense spatiotemporal grounding coupled with open-web exploration. Preliminary evaluations reveal two critical bottlenecks in current models: (1) modality bias, where agents bypass visual tools in favor of textual search, and (2) parametric knowledge leakage, where models rely on internal memory rather than genuine tool-augmented execution. To address these challenges, we propose Video-DeepResearch, featuring a decoupled perception-exploration pipeline with stage-wise tool unlocking that compels exhaustive cross-frame visual grounding prior to web retrieval. Our framework adopts a two-stage training recipe—supervised fine-tuning followed by Group Relative Policy Optimization (GRPO)—enabling autonomous exploration that breaks the imitation-learning ceiling. Furthermore, we curate VideoDR-Bench, a human-AI collaborative benchmark comprising 200 complex, multi-hop VQA instances. Empirical results demonstrate that our Video-DeepResearch-35B-A3B establishes a new state-of-the-art of 64.0% average accuracy, surpassing proprietary Claude-4.5-Sonnet (59.0%) by 5.0 points and significantly outperforming GPT-5 (52.5%) and Gemini 2.5 Pro (57.5%). The 30B-A3B variant achieves 59.3%, competitive with Claude-4.5-Sonnet and demonstrating the effectiveness of our training paradigm even at compact scale.
NüshuVoice: Reviving the Voice of Endangered Nüshu with Pitch-Aware Text-to-Speech
Hongkun Yang | YI Xinhui | Zhao Xiyan | Yibo Meng | Lionel Z. Wang | Lixu Wang | Yaqi Zhang | Ruiqi Chen | Xuanyue Zhao | Lanxin Zhang | Yu Zeng | Chu Weijia | Yiming Ma | Chenyu Liu | Jianghao Lin | Xu Xin
Findings of the Association for Computational Linguistics: IJCNLP-AACL 2026
Hongkun Yang | YI Xinhui | Zhao Xiyan | Yibo Meng | Lionel Z. Wang | Lixu Wang | Yaqi Zhang | Ruiqi Chen | Xuanyue Zhao | Lanxin Zhang | Yu Zeng | Chu Weijia | Yiming Ma | Chenyu Liu | Jianghao Lin | Xu Xin
Findings of the Association for Computational Linguistics: IJCNLP-AACL 2026
Nüshu is an endangered phonetic script historically used by women in Jiangyong County, southern Hunan, China. While existing computational studies of Nüshu mainly focus on textual digitization and visual recognition, the acoustic reconstruction of its authentic pronunciation remains largely unexplored. Building a Nüshu text-to-speech (TTS) system is particularly challenging because available recordings are extremely limited and mostly consist of isolated syllable-level pronunciations rather than natural sentence-level utterances. In this work, we introduce NüshuVoice, the first TTS benchmark for Nüshu. We construct a sentence-level Nüshu text-to-audio dataset that aligns standardized Unicode Nüshu text, phonetic transcriptions, standard Chinese translations, and archival recordings. To synthesize speech under this extreme low-resource setting, we propose Nüshu-PitchVITS, an -conditioned VITS framework that leverages Nüshu’s five-level pitch notation as an explicit prosodic inductive bias. Experimental results show that Nüshu-PitchVITS outperforms strong TTS baselines in spectral fidelity, pitch reconstruction, and human-rated intelligibility. We publicly release the dataset and code at: https://anonymous.4open.science/r/Nvshu-TTS-2EB6.
UniCorn: Towards Self-Improving Unified Multimodal Models through Self-Generated Supervision
Zhen Fang | Ruiyan Han | XinYu Sun | Yuchen Ma | Ziheng Wang | Yu Zeng | Zehui Chen | Lin Chen | Wenxuan Huang | Wei-Jie Xu | Yi Cao | Feng Zhao
Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
Zhen Fang | Ruiyan Han | XinYu Sun | Yuchen Ma | Ziheng Wang | Yu Zeng | Zehui Chen | Lin Chen | Wenxuan Huang | Wei-Jie Xu | Yi Cao | Feng Zhao
Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
While Unified Multimodal Models (UMMs) have achieved remarkable success in cross-modal comprehension, a significant gap persists in their ability to leverage such internal knowledge for high-quality generation. We formalize this discrepancy as Conduction Aphasia, a phenomenon where models accurately interpret multimodal inputs but struggle to translate that understanding into faithful and controllable synthesis. To address this, we propose UniCorn, a simple yet elegant self-improvement framework that eliminates the need for external data or teacher supervision. By partitioning a single UMM into three collaborative roles: Proposer, Solver, and Judge, UniCorn generates high-quality interactions via self-play and employs cognitive pattern reconstruction to distill latent understanding into explicit generative signals. To validate the restoration of multimodal coherence, we introduce UniCycle, a cycle-consistency benchmark based on a Text to Image to Text reconstruction loop. Extensive experiments demonstrate that UniCorn achieves comprehensive and substantial improvements over the base model across six general image generation benchmarks. Notably, it achieves SOTA performance on TIIF(73.8), DPG(86.8), CompBench(88.5), and UniCycle while further delivering substantial gains of +5.0 on WISE and +6.5 on OneIG. These results highlight that our method significantly enhances T2I generation while maintaining robust comprehension, demonstrating the scalability of fully self-supervised refinement for unified multimodal intelligence.
2025
Enhancing Large Vision-Language Models with Ultra-Detailed Image Caption Generation
Yu Zeng | Yukun Qi | Yiming Zhao | Xikun Bao | Lin Chen | Zehui Chen | Shiting Huang | Jie Zhao | Feng Zhao
Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing
Yu Zeng | Yukun Qi | Yiming Zhao | Xikun Bao | Lin Chen | Zehui Chen | Shiting Huang | Jie Zhao | Feng Zhao
Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing
High-quality image captions are essential for improving modality alignment and visual understanding in Large Vision-Language Models (LVLMs). However, the scarcity of ultra-detailed image caption data limits further advancements. This paper presents a systematic pipeline for generating high-quality, ultra-detailed image captions, encompassing both pre-processing and post-processing stages. In the pre-processing stage, we classify and deduplicate images, extract visual information using expert tools, and leverage GPT-4o with structured prompts to generate initial captions. To enhance comprehensiveness, we introduce an expansion strategy based on Large Language Models (LLMs), defining eight descriptive dimensions to refine and extend captions, which serve as seed data for training a proprietary captioner model. In the post-processing stage, we incorporate human error-correction annotations and an active learning-inspired approach to refine low-quality samples. Using high-quality corrected data, we apply Direct Preference Optimization (DPO) and develop a critic-rewrite pipeline, training a sentence-level critic model to mitigate hallucinations. Experimental results demonstrate that our ultra-detailed captions significantly enhance LVLMs’ perception and cognitive abilities across multiple vision-language benchmarks. The code and dataset are available at https://github.com/yuzeng0-0/UltraCaption.
CRITICTOOL: Evaluating Self-Critique Capabilities of Large Language Models in Tool-Calling Error Scenarios
Shiting Huang | Zhen Fang | Zehui Chen | Siyu Yuan | Junjie Ye | Yu Zeng | Lin Chen | Qi Mao | Feng Zhao
Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing
Shiting Huang | Zhen Fang | Zehui Chen | Siyu Yuan | Junjie Ye | Yu Zeng | Lin Chen | Qi Mao | Feng Zhao
Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing
The ability of large language models (LLMs) to utilize external tools has enabled them to tackle an increasingly diverse range of tasks. However, as the tasks become more complex and long-horizon, the intricate tool utilization process may trigger various unexpected errors. Therefore, how to effectively handle such errors, including identifying, diagnosing, and recovering from them, has emerged as a key research direction for advancing tool learning. In this work, we first extensively analyze the types of errors encountered during the function-calling process on several competitive tool evaluation benchmarks. Based on it, we introduce CRITICTOOL, a comprehensive critique evaluation benchmark specialized for tool learning. Building upon a novel evolutionary strategy for dataset construction, CRITICTOOL holds diverse tool-use errors with varying complexities, which better reflects real-world scenarios. We conduct extensive experiments on CRITICTOOL, and validate the generalization and effectiveness of our constructed benchmark strategy. We also provide an in-depth analysis of the tool reflection ability on various LLMs, offering a new perspective on the field of tool learning in LLMs. The code is available at https://github.com/Shellorley0513/CriticTool.
Search
Fix author
Co-authors
- Lin Chen 5
- Zehui Chen 5
- Feng Zhao 5
- Zhen Fang 4
- Shiting Huang 4
- Wenxuan Huang 2
- Qingnan Ren 2
- Qisheng Su 2
- Lionel Z. Wang 2
- Yiming Zhao 2
- Xikun Bao 1
- Shaosheng Cao 1
- Yi Cao 1
- Ruiqi Chen 1
- Shuang Chen 1
- Ruiyan Han 1
- Yao Hu 1
- Zecheng Li 1
- Jianghao Lin 1
- Shaohui Lin 1
- Chenyu Liu 1
- Qi Lu 1
- Yiming Ma 1
- Yuchen Ma 1
- Qi Mao 1
- Yibo Meng 1
- Wanli Ouyang 1
- Yukun Qi 1
- Tianfei Ren 1
- Kou Shi 1
- XinYu Sun 1
- Lixu Wang 1
- Ziheng Wang 1
- Chu Weijia 1
- Xu Xin 1
- YI Xinhui 1
- Zhao Xiyan 1
- Wei-Jie Xu 1
- Hongkun Yang 1
- Junjie Ye (叶俊杰) 1
- Qingyu Yin 1
- Zhenfei Yin 1
- Siyu Yuan 1
- Lanxin Zhang 1
- Yaqi Zhang 1
- Ziao Zhang 1
- Jie Zhao 1
- Xuanyue Zhao 1