Yiwen Song
Author directory2026
CANVAS: Continuity-Aware Narrative Generation via Visual Agentic Storyboarding
Ishani Mondal | Yiwen Song | Mihir Parmar | Palash Goyal | Jordan Boyd-Graber | Tomas Pfister | Yale Song
Findings of the Association for Computational Linguistics: EMNLP 2026
Ishani Mondal | Yiwen Song | Mihir Parmar | Palash Goyal | Jordan Boyd-Graber | Tomas Pfister | Yale Song
Findings of the Association for Computational Linguistics: EMNLP 2026
Long-form visual storytelling requires maintaining continuity across shots, including consistent characters, stable environments, and smooth scene transitions. While existing generative models can produce strong individual frames, they fail to preserve such continuity, leading to appearance changes, inconsistent backgrounds, and abrupt scene shifts. We introduce CANVAS (Continuity-Aware Narratives via Visual Agentic Storyboarding), a multi-agent framework that explicitly plans visual continuity in multi-shot narratives. CANVAS enforces coherence through character continuity, persistent background anchors, and location-aware scene planning for smooth transitions within the same setting. We evaluate CANVAS on two storyboard generation benchmarks ST-BENCH and ViStoryBench and introduce a new challenging benchmark HardContinuityBench for long-range narrative consistency. CANVAS consistently outperforms the best-performing baseline, improving background continuity by 21.6%, character consistency by 9.6% and props consistency by 7.6%.
Judging with Confidence: Calibrating Autoraters to Preference Distributions
Zhuohang Li | Xiaowei Li | Chengyu Huang | Guowang Li | Katayoon Goshvadi | Bo Dai | Dale Schuurmans | Xin Zhou | Hamid Palangi | Yiwen Song | Palash Goyal | Murat Kantarcioglu | Bradley A. Malin | Yuan Xue
Findings of the Association for Computational Linguistics: EMNLP 2026
Zhuohang Li | Xiaowei Li | Chengyu Huang | Guowang Li | Katayoon Goshvadi | Bo Dai | Dale Schuurmans | Xin Zhou | Hamid Palangi | Yiwen Song | Palash Goyal | Murat Kantarcioglu | Bradley A. Malin | Yuan Xue
Findings of the Association for Computational Linguistics: EMNLP 2026
The alignment of large language models (LLMs) with human values increasingly relies on using other LLMs as automated judges, or "autoraters". However, their reliability is limited by a foundational issue: they are trained on discrete preference labels, forcing a single ground truth onto tasks that are often subjective, ambiguous, or nuanced. We argue that a reliable autorater must learn to model the full distribution of preferences defined by a target population. In this paper, we propose a general framework for calibrating probabilistic autoraters to any given preference distribution. We formalize the problem and present two learning methods tailored to different data conditions: 1) a direct supervised fine-tuning for dense, probabilistic labels, and 2) a reinforcement learning approach for sparse, binary labels. Our empirical results show that finetuning autoraters with a distribution-matching objective leads to verbalized probability predictions that are better aligned with the target preference distribution, with improved calibration and significantly lower positional bias, all while preserving performance on objective tasks.
2025
PLAN-TUNING: Post-Training Language Models to Learn Step-by-Step Planning for Complex Problem Solving
Mihir Parmar | Palash Goyal | Xin Liu | Yiwen Song | Mingyang Ling | Chitta Baral | Hamid Palangi | Tomas Pfister
Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing
Mihir Parmar | Palash Goyal | Xin Liu | Yiwen Song | Mingyang Ling | Chitta Baral | Hamid Palangi | Tomas Pfister
Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing
Recently, decomposing complex problems into simple subtasks–a crucial part of human-like natural planning–to solve the given problem has significantly boosted the performance of large language models (LLMs). However, leveraging such planning structures during post-training to boost the performance of smaller open-source LLMs remains underexplored. Motivated by this, we introduce PLAN-TUNING, a unified post-training framework that (i) distills synthetic task decompositions (termed “planning trajectories”) from large-scale LLMs and (ii) fine-tunes smaller models via supervised and reinforcement-learning objectives designed to mimic these planning processes to improve complex reasoning. On GSM8k and the MATH benchmarks, plan-tuned models outperform strong baselines by an average ~7%. Furthermore, plan-tuned models show better generalization capabilities on out-of-domain datasets, with average ~10% and ~12% performance improvements on OlympiadBench and AIME 2024, respectively. Our detailed analysis demonstrates how planning trajectories improves complex reasoning capabilities, showing that PLAN-TUNING is an effective strategy for improving task-specific performance of smaller LLMs.
In Prospect and Retrospect: Reflective Memory Management for Long-term Personalized Dialogue Agents
Zhen Tan | Jun Yan | I-Hung Hsu | Rujun Han | Zifeng Wang | Long Le | Yiwen Song | Yanfei Chen | Hamid Palangi | George Lee | Anand Rajan Iyer | Tianlong Chen | Huan Liu | Chen-Yu Lee | Tomas Pfister
Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
Zhen Tan | Jun Yan | I-Hung Hsu | Rujun Han | Zifeng Wang | Long Le | Yiwen Song | Yanfei Chen | Hamid Palangi | George Lee | Anand Rajan Iyer | Tianlong Chen | Huan Liu | Chen-Yu Lee | Tomas Pfister
Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
Large Language Models (LLMs) have made significant progress in open-ended dialogue, yet their inability to retain and retrieve relevant information from long-term interactions limits their effectiveness in applications requiring sustained personalization. External memory mechanisms have been proposed to address this limitation, enabling LLMs to maintain conversational continuity. However, existing approaches struggle with two key challenges. First, rigid memory granularity fails to capture the natural semantic structure of conversations, leading to fragmented and incomplete representations. Second, fixed retrieval mechanisms cannot adapt to diverse dialogue contexts and user interaction patterns. In this work, we propose Reflective Memory Management (RMM), a novel mechanism for long-term dialogue agents, integrating forward- and backward-looking reflections: (1) Prospective Reflection, which dynamically summarizes interactions across granularities—utterances, turns, and sessions—into a personalized memory bank for effective future retrieval, and (2) Retrospective Reflection, which iteratively refines the retrieval in an online reinforcement learning (RL) manner based on LLMs’ cited evidence. Experiments show that RMM demonstrates consistent improvement across various metrics and benchmarks. For example, RMM shows more than 10% accuracy improvement over the baseline without memory management on the LongMemEval dataset.
Search
Fix author
Co-authors
- Palash Goyal 3
- Hamid Palangi 3
- Tomas Pfister 3
- Mihir Parmar 2
- Chitta Baral 1
- Jordan Lee Boyd-Graber 1
- Tianlong Chen 1
- Yanfei Chen 1
- Bo Dai 1
- Katayoon Goshvadi 1
- Rujun Han 1
- I-Hung Hsu 1
- Chengyu Huang 1
- Anand Rajan Iyer 1
- Murat Kantarcioglu 1
- Long Le 1
- Chen-Yu Lee 1
- George Lee 1
- Guowang Li 1
- Xiaowei Li 1
- Zhuohang Li 1
- Mingyang Ling 1
- Huan Liu 1
- Xin Liu 1
- Bradley A. Malin 1
- Ishani Mondal 1
- Dale Schuurmans 1
- Yale Song 1
- Zhen Tan 1
- Zifeng Wang 1
- Yuan Xue 1
- Jun Yan 1
- Xin Zhou 1