Ming Sun
Author directory2026
MMG2Skill: Can Agents Distill In-the-Wild Guides into Self-Evolving Skills?
Xinyu Che | Junqi Xiong | Yunfei Ge | Xinping Lei | Shihao Li | Hang Yan | Han Li | Yuanxing Zhang | Zhiqi Bai | Jinghua Hao | Ming Sun | Han Li | Jiaheng Liu
Findings of the Association for Computational Linguistics: EMNLP 2026
Xinyu Che | Junqi Xiong | Yunfei Ge | Xinping Lei | Shihao Li | Hang Yan | Han Li | Yuanxing Zhang | Zhiqi Bai | Jinghua Hao | Ming Sun | Han Li | Jiaheng Liu
Findings of the Association for Computational Linguistics: EMNLP 2026
Abundant procedural knowledge on the Web holds great potential for helping agents solve long-horizon tasks. However, such knowledge is often multimodal, heterogeneous, noisy, and implicitly assumes human executors, making it difficult to use directly as the skills required by agents. To bridge the gap between human-oriented guides and agent-executable skills, we formalize this problem as guide-to-skill learning: converting in-the-wild guides into executable skills and continuously improving them from trajectories observable to the agent. To evaluate the capability of existing agents on this task, we introduce MMG2Skill-Bench, the first benchmark designed for this problem. We further propose MMG2Skill, a closed-loop framework that compiles guides into editable skills, conditions a fixed vision-language model (VLM) agent on these skills during execution, and revises the skills from trajectory-level root-cause feedback without using benchmark scores. Across GUI control, open-ended gameplay, and strategic card play with six VLM backbones, MMG2Skill consistently outperforms vanilla baseline agents in every model–domain setting, achieving macro-average gains of +12.8 to +25.3 percentage points across backbones. Ablation studies show that directly prompting agents with raw guides can degrade performance, while both structured skill construction and trajectory-driven revision are necessary for the observed improvements. On success-inferable tasks, analyzer-based early stopping further prevents late-stage performance regressions and saves 25%–53% of attempts when the success signal is properly calibrated.
AgendaFlow: Task-Adaptive Meeting Protocols for Multi-Agent LLM Reasoning
Ming Sun | Yin Wang | Yanlin Zhang
Findings of the Association for Computational Linguistics: EMNLP 2026
Ming Sun | Yin Wang | Yanlin Zhang
Findings of the Association for Computational Linguistics: EMNLP 2026
Multi-agent large language model (LLM) systems often rely on independent solvers, critique, debate, or answer aggregation. While these designs increase diversity and feedback, they do not necessarily create effective group reasoning: information may be poorly shared, expertise underused, discussions may drift, and consensus may emerge before unresolved issues are examined. We introduce AgendaFlow, a meeting-science-inspired framework that treats multi-agent reasoning as a structured meeting rather than parallel answering or generic debate. AgendaFlow routes each task to a suitable agenda, instantiates complementary functional roles, and uses a moderator to manage phases, speaking order, shared artifacts, unresolved issues, and completion checks. This enables agents to pool information, surface constraints, compare alternatives, repair errors, and converge through a managed collective decision process. Across five reasoning, planning, and rule- or evidence-based benchmarks, AgendaFlow consistently outperforms single-agent, sampling-based, debate-style, and multi-agent orchestration baselines. Ablations, token-matched comparisons, and process-level analyses suggest that the gains come from meeting-level organization rather than agent diversity or additional computation alone, highlighting meeting design as a missing dimension in multi-agent LLM systems. Code is available at: https://github.com/sunming-code/agendaflow.
Riemannian Large Language Modeling
Shu XU | Jiahong Liu | Wenhao Zhuang | Jiabao li | Yefan Wang | Ming Sun | Menglin Yang
Findings of the Association for Computational Linguistics: EMNLP 2026
Shu XU | Jiahong Liu | Wenhao Zhuang | Jiabao li | Yefan Wang | Ming Sun | Menglin Yang
Findings of the Association for Computational Linguistics: EMNLP 2026
Current large language models perform next-token prediction under the implicit assumption that all tokens lie in a shared Euclidean embedding space, which treats every token uniformly. However, natural language contains rich structural relationships, including hierarchical, compositional, and cyclic patterns, which the flat geometry of Euclidean space cannot model well. To address this limitation, we propose rLLM (Riemannian Large Language Modeling), which introduces a token-adaptive positive-definite diagonal metric tensor into the LM head. This metric dynamically adjusts the local quadratic score for each hidden state and its context, allowing the model to better capture the relationships among tokens and prompts, with less than 0.01% additional parameters. We train rLLM from scratch across three different scales (2.8B to 5.0B parameters), covering the full pretraining, SFT, and RL pipeline. Extensive experiments show that rLLM improves Base CORE, a composite of 22 in-context learning tasks, by up to +2.5pp at the two smaller GPT scales and obtains higher ChatCORE at all three scales, with gains up to +3.0pp after RL. These results suggest that revisiting the geometric assumptions underlying token representations is a promising direction for building more expressive and capable language models. Code is available at https://github.com/digailab/riemannianllm.
Validity Is Not Visibility: A Lifecycle FSM for Agent Memory
Shuo Zhou | Pengcheng Xu | Shaotong qi | Li Que | Ming Sun | Han Li
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Shuo Zhou | Pengcheng Xu | Shaotong qi | Li Que | Ming Sun | Han Li
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Long-term memory for LLM agents must answer two questions for every stored fact: whether the fact is still true and whether it should be surfaced for a given query. We refer to these two dimensions as validity and visibility, respectively. Existing memory systems often collapse these two decisions into a single scalar score. However, current and historical queries may require opposite retrieval decisions for the same fact: a superseded fact should be hidden for a current-intent query but surfaced for a historical-intent one. This creates a structural failure mode: superseded and expired facts are collapsed onto the same validity axis. To address this problem, we propose MemFSM, a training-free retrieval-side layer that separates validity from visibility through a memory-lifecycle finite-state machine. Along the validity axis, a four-state FSM tracks whether each fact has been superseded and expired. Along the visibility axis, an intent-aware router decides whether to surface or hide a fact based on its state and the query intent. By decoupling visibility from validity, MemFSM improves performance across heterogeneous memory benchmarks and diverse LLM backbones. To evaluate memory systems under fact conflicts and diverse query intents, we further introduce ValidMem, a benchmark that annotates each fact with both supersedure and expiry labels and pairs each memory store with both current-intent and historical-intent queries.
WebCompass: Evaluating Multimodal Code Agents in Web Development from Vision, Instruction, and Interaction to Implementation
Xinping Lei | Xinyu Che | Junqi Xiong | Chenchen Zhang | Yukai Huang | Chenyu Zhou | Haoyang Huang | Minghao Liu | Letian Zhu | Hongyi Ye | Ken Deng | Zizheng Zhan | Han Li | Yifan Yao | Ming Sun | Zhaoxiang Zhang | Jiaheng Liu
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Xinping Lei | Xinyu Che | Junqi Xiong | Chenchen Zhang | Yukai Huang | Chenyu Zhou | Haoyang Huang | Minghao Liu | Letian Zhu | Hongyi Ye | Ken Deng | Zizheng Zhan | Han Li | Yifan Yao | Ming Sun | Zhaoxiang Zhang | Jiaheng Liu
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Large language models are increasingly deployed as interactive coding agents for end-to-end web development, yet existing benchmarks evaluate narrow slices of this workflow—typically text-conditioned generation with static correctness metrics—overlooking visual fidelity, interactivity, and codebase-level reasoning. We introduce WebCompass, a benchmark that covers the full web development lifecycle by spanning three input modalities (text, image, video) and three task types (generation, editing, repair). To match the diverse nature of these tasks, we pair checklist-guided LLM-as-a-Judge for patch-based evaluation with a novel Agent-as-a-Judge protocol that executes generated websites in a real browser and synthesizes targeted test cases. Experiments on ten models reveal a 25-point gap between the best closed- and open-source models, persistent weaknesses in visual aesthetics even among frontier systems, and divergent performance profiles across task types that challenge the notion of a single "best" web coding agent.
2019
Neural Text Normalization with Subword Units
Courtney Mansfield | Ming Sun | Yuzong Liu | Ankur Gandhe | Björn Hoffmeister
Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Industry Papers)
Courtney Mansfield | Ming Sun | Yuzong Liu | Ankur Gandhe | Björn Hoffmeister
Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Industry Papers)
Text normalization (TN) is an important step in conversational systems. It converts written text to its spoken form to facilitate speech recognition, natural language understanding and text-to-speech synthesis. Finite state transducers (FSTs) are commonly used to build grammars that handle text normalization. However, translating linguistic knowledge into grammars requires extensive effort. In this paper, we frame TN as a machine translation task and tackle it with sequence-to-sequence (seq2seq) models. Previous research focuses on normalizing a word (or phrase) with the help of limited word-level context, while our approach directly normalizes full sentences. We find subword models with additional linguistic features yield the best performance (with a word error rate of 0.17%).
2016
AppDialogue: Multi-App Dialogues for Intelligent Assistants
Ming Sun | Yun-Nung Chen | Zhenhao Hua | Yulian Tamres-Rudnicky | Arnab Dash | Alexander Rudnicky
Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC'16)
Ming Sun | Yun-Nung Chen | Zhenhao Hua | Yulian Tamres-Rudnicky | Arnab Dash | Alexander Rudnicky
Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC'16)
Users will interact with an individual app on smart devices (e.g., phone, TV, car) to fulfill a specific goal (e.g. find a photographer), but users may also pursue more complex tasks that will span multiple domains and apps (e.g. plan a wedding ceremony). Planning and executing such multi-app tasks are typically managed by users, considering the required global context awareness. To investigate how users arrange domains/apps to fulfill complex tasks in their daily life, we conducted a user study on 14 participants to collect such data from their Android smart phones. This document 1) summarizes the techniques used in the data collection and 2) provides a brief statistical description of the data. This data guilds the future direction for researchers in the fields of conversational agent and personal assistant, etc. This data is available at http://AppDialogue.com.
2014
Search
Fix author
Co-authors
- Xinyu Che 2
- Xinping Lei 2
- Han Li 2
- Han Li 2
- Jiaheng Liu 2
- Alexander Rudnicky 2
- Junqi Xiong 2
- Zhiqi Bai 1
- Yun-Nung Chen 1
- Arnab Dash 1
- Ken Deng 1
- Ankur Gandhe 1
- Yunfei Ge 1
- Jinghua Hao 1
- Björn Hoffmeister 1
- Zhenhao Hua 1
- Haoyang Huang 1
- Yukai Huang 1
- Shihao Li 1
- Jiahong Liu 1
- Minghao Liu 1
- Yuzong Liu 1
- Courtney Mansfield 1
- Aasish Pappu 1
- Li Que 1
- Seshadri Sridharan 1
- Yulian Tamres-Rudnicky 1
- Yefan Wang 1
- Yin Wang 1
- Shu XU 1
- Pengcheng Xu 1
- Hang Yan 1
- Menglin Yang 1
- Yifan Yao 1
- Hongyi Ye 1
- Zizheng Zhan 1
- Chenchen Zhang 1
- Yanlin Zhang 1
- Yuanxing Zhang 1
- Zhaoxiang Zhang 1
- Chenyu Zhou 1
- Shuo Zhou 1
- Letian Zhu 1
- Wenhao Zhuang (庄文浩) 1
- Jiabao li 1
- Shaotong qi 1