Nanjie Li
Author directory2026
Self-Corrected Speech Reasoning: Diagnosing and Improving Condition Updating via PACER
Zijie Zeng | Liwen Shi | Yunji | Bowen Wang | Nanjie Li | Xiaoyong Guo | Zeyu Chen | Xinzheng Sun | Xuyang Xiong | Haoyang Wu | Sheng Zhang
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Zijie Zeng | Liwen Shi | Yunji | Bowen Wang | Nanjie Li | Xiaoyong Guo | Zeyu Chen | Xinzheng Sun | Xuyang Xiong | Haoyang Wu | Sheng Zhang
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Speech large language models (Speech LLMs) are increasingly expected to reason in everyday spoken interactions, where users often revise task conditions through self-correction. Unlike ordinary disfluencies, self-corrections alter the premise structure of a problem: obsolete conditions must be discarded, while final corrected conditions must guide reasoning. We formulate this setting as self-corrected speech reasoning and study it as a condition-update problem. We construct SCUR-Bench (Spoken Condition-Update Reasoning Benchmark), a controlled Chinese benchmark of 13,295 instances organized by reasoning difficulty and self-correction difficulty, and introduce a three-path diagnostic protocol to analyze speech-side recovery, condition use, and reasoning. Our analysis reveals a recurrent failure pattern in current Speech LLMs: corrected conditions may be perceived, transcribed, or even mentioned, yet still fail to serve as active premises in reasoning. We term this final-condition adoption failure. A controlled English parallel evaluation exhibits the same recovery–adoption gap, providing preliminary evidence that this failure is not unique to Chinese. To mitigate it, we propose PACER, a perception-aware condition-enhanced reinforcement framework based on GRPO. On Qwen2.5-Omni-7B, PACER reduces end-to-end error and condition-related errors by 39.8% and 67.5%, respectively, relative to the untrained baseline.
LCMA-SRT: Language-Conditional Mixture-of-Experts Adapters for Joint Multilingual Speech Recognition and Translation
Nanjie Li | Xiaoyong Guo | Hao Huang | Xu Haihua | Wei Shi
Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
Nanjie Li | Xiaoyong Guo | Hao Huang | Xu Haihua | Wei Shi
Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
Neural transducers offer an alignment-free framework for speech-to-text modeling, and hierarchical transducer architectures further improve multilingual joint automatic speech recognition (ASR) and speech translation (ST) by stacking a translation-focused encoder on top of an ASR encoder. However, extending hierarchical transducers to multilingual many-to-many settings remains challenging: fully shared models often suffer from negative transfer and unstable target-language generation, while training separate models for each direction is computationally prohibitive. We propose LCMA-SRT (Language-Conditional Mixture-of-Experts Adapters for Speech Recognition and Translation), which augments a hierarchical transducer with language-conditional Mixture-of-Experts (MoE) adapters. A source-conditioned MoE adapter (SRC-MoE) uses source-language embeddings to reduce cross-language interference and improve multilingual ASR. A target-conditioned MoE adapter (TGT-MoE) uses the desired target language to reduce cross-target interference and stabilize target-language generation in many-to-many ST. Experiments on Europarl-ST (9 languages, 72 directions) show that LCMA-SRT improves both ASR and ST within a single joint model, reducing average WER and improving BLEU and COMET over strong hierarchical transducer baselines. We release our code and models at https://github.com/linanjie0820/LCMA-SRT.