Ruixiang Mao
Author directory2026
When Scaling Fails: Mitigating Audio Perception Decay of LALMs via Multi-Step Perception-Aware Reasoning
Ruixiang Mao | Xiangnan Ma | Dan Chen | Ziming Zhu | Yuan Ge | Haishu Zhao | AoKai Hao | Yifu Huo | Qing Yang | Kaiyan Chang | Xiaoqian Liu | Chenglong Wang | Weiqiao Shan | Qiaozhi He | Changliang Li | Zhengtao Yu | Tong Xiao | JingBo Zhu
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Ruixiang Mao | Xiangnan Ma | Dan Chen | Ziming Zhu | Yuan Ge | Haishu Zhao | AoKai Hao | Yifu Huo | Qing Yang | Kaiyan Chang | Xiaoqian Liu | Chenglong Wang | Weiqiao Shan | Qiaozhi He | Changliang Li | Zhengtao Yu | Tong Xiao | JingBo Zhu
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Test-Time Scaling has shown notable efficacy in addressing complex problems through scaling inference compute. However, within Large Audio-Language Models (LALMs), an unintuitive phenomenon exists: post-training models for structured reasoning trajectories results in marginal or even negative gains compared to training for direct answering. To investigate it, we introduce **CAFE**, an evaluation framework designed to quantify audio reasoning errors. Evaluation results reveal that LALMs encounter a critical bottleneck: **Audio Perception Decay**, which compromises reasoning performance. To address it, we propose **MPAR2**, a strategy that encourages a step-by-step reasoning process with rich perceptual awareness. With reinforcement learning, MPAR2 improves performance on CAFE from 31.74% to 63.51% and effectively mitigates audio perception decay, concurrently enhancing reasoning ability to achieve a strong 74.59% accuracy on the MMAU benchmark. Further analysis shows that MPAR2 optimizes audio attention and enhances reasoning token budget efficiency.
SPRI: SVD-Partitioned Residual Initialization for Data-Constrained MoE Upcycling in Speech-to-Text Translation
Weiqiao Shan | Ruixiang Mao | Yuang Li | Yuhao Zhang | Yingfeng Luo | Tong Zheng | Chen Xu | Yucheng Qiao | Chunxiang Jin | Yi Yuan | Jingdong Chen | Tong Xiao | JingBo Zhu
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Weiqiao Shan | Ruixiang Mao | Yuang Li | Yuhao Zhang | Yingfeng Luo | Tong Zheng | Chen Xu | Yucheng Qiao | Chunxiang Jin | Yi Yuan | Jingdong Chen | Tong Xiao | JingBo Zhu
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Mixture-of-Experts (MoE) models enable efficient scaling, but training them from scratch remains prohibitively expensive. MoE upcycling mitigates this cost by converting pretrained dense models into sparse MoE models. However, existing upcycling methods typically rely on large-scale continued training and often perform poorly under data-constrained supervised adaptation, due to either homogeneous experts or overly disruptive perturbations to pretrained parameters. In this setting, effective upcycling must leverage pretrained weight structure while introducing sufficient diversity among routed experts. To this end, we propose SVD-Partitioned Residual Initialization (SPRI), which distributes SVD-partitioned residuals derived from pretrained feed-forward network (FFN) weights across routed experts, introducing controlled expert diversity grounded in pretrained spectral structure. We further introduce a two-stage training strategy to improve adaptation stability. We evaluate SPRI on multilingual speech-to-text translation, where limited supervised data challenges MoE upcycling and multiple target languages provide natural routing heterogeneity. On CoVoST2 across 15 En-to-X directions, SPRI improves average BLEU and COMET over fully fine-tuned dense models by 2.58 and 3.32 points, respectively, and outperforms the strongest prior MoE upcycling method by 2.44 BLEU and 3.21 COMET points. Our code is publicly available at https://github.com/shanweiqiao/SPRI.