Yu Xi
Author directory2026
TC-BiMamba: Efficient Dynamic-Chunk Bidirectional Mamba for Unified Streaming and Offline ASR
Jing Peng | Qingshun She | Yangui Fang | Yu Xi | Shuai Wang | Kai Yu
Findings of the Association for Computational Linguistics: EMNLP 2026
Jing Peng | Qingshun She | Yangui Fang | Yu Xi | Shuai Wang | Kai Yu
Findings of the Association for Computational Linguistics: EMNLP 2026
Unified streaming and offline ASR aims to deliver full-context accuracy and low-latency incremental decoding with a single model. Transformer/Conformer systems are effective, but long-form streaming inference still suffers from growing attention masks, historical caches, and memory states. Latency-Controlled BiMamba (LC-BiMamba) offers attention-free bidirectional modeling, yet its backward branch is confined to fixed chunks, tying the model to one latency. Extending it to dynamic chunk training expands mini-batches and slows training. We propose TC-BiMamba, whose Trans-Chunk formulation casts local-future backward modeling as sequence-level dynamic-chunk training over complete sequences. TC-BiMamba supports multiple chunk-wise latency settings, keeps the encoder/CTC path memory-stable on long-form inputs, and improves training scalability. On AISHELL-1, AISHELL-2, and LibriSpeech, it outperforms U2++ and remains competitive with externally reported LC-BiMamba results using fewer parameters, while achieving 1.3 times training speedup and 50% GPU memory reduction.
G-STAR: End-to-End Global Speaker-Tracking Attributed Recognition
Jing Peng | Ziyi Chen | Haoyu Li | Yucheng Wang | Mengtian Li | Duo MA | Yu Xi | Yunfan Du | Kai Yu | Shuai Wang
Findings of the Association for Computational Linguistics: EMNLP 2026
Jing Peng | Ziyi Chen | Haoyu Li | Yucheng Wang | Mengtian Li | Duo MA | Yu Xi | Yunfan Du | Kai Yu | Shuai Wang
Findings of the Association for Computational Linguistics: EMNLP 2026
We study timestamped speaker-attributed automatic speech recognition (SA-ASR) for long-form, multi-party speech with overlap. In this setting, chunk-wise inference must preserve meeting-level speaker identity consistency while producing time-stamped, speaker-labeled transcripts. Prior Speech-LLM systems tend to prioritize either local diarization or global labeling, lacking the ability to jointly model fine-grained temporal boundaries and robust cross-chunk identity linking. We propose G-STAR, an end-to-end framework that couples a cache-conditioned speaker-tracking module with a Speech-LLM transcription backbone. The tracker provides structured speaker cues with temporal grounding, and the LLM generates attributed text conditioned on these cues. G-STAR supports component-wise optimization and joint end-to-end training, enabling flexible learning under heterogeneous supervision and domain shift. Under chunk-wise decoding protocols, experiments on both oracle-segmented local evaluation and full-meeting global evaluation show strong speaker-attributed transcription performance.
HoliTok: A Continuous Holistic Tokenization with Robust Dual Capabilities of Speech Generation and Understanding
Bohan Li | Lianshi | Hankun Wang | Yiwei Guo | Yu Xi | Zhihan Li | Da Zheng | Colin Zhang | Kai Yu
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Bohan Li | Lianshi | Hankun Wang | Yiwei Guo | Yu Xi | Zhihan Li | Da Zheng | Colin Zhang | Kai Yu
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Unified speech foundation models require a holistic tokenization space that is both learnable by language models and decodable into high-quality waveforms. Existing speech tokenizers, however, often fail to satisfy these requirements simultaneously, leading to increased architectural complexity and more involved training designs. We propose HoliTok, a continuous Holistic speech Tokenization model designed for unified generation-understanding modeling. HoliTok encodes 48kHz speech into a compact 25Hz sequence of 128-dimensional latents. It is trained with a progressive strategy that jointly preserves signal-level fidelity, incorporates semantic information, and maintains strong latent learnability. Based on this tokenization, we build a unified AR+DiT model for speech synthesis and recognition, where the same latent sequence supports both generation-specific and unified generation-understanding tasks. Experiments show that HoliTok achieves competitive reconstruction fidelity, improves generative learnability for high-quality and controllable synthesis, and, among the evaluated representations, is the only one that operates robustly in our unified generation-understanding architecture without additional optimization tricks. These results suggest that HoliTok serves as an effective speech tokenizer and a foundational representation interface for unified spoken language modeling. The code is available at: https://anonymous.4open.science/r/HoliTok-2852.
2025
UniCodec: Unified Audio Codec with Single Domain-Adaptive Codebook
Yidi Jiang | Qian Chen | Shengpeng Ji | Yu Xi | Wen Wang | Chong Zhang | Xianghu Yue | ShiLiang Zhang | Haizhou Li
Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
Yidi Jiang | Qian Chen | Shengpeng Ji | Yu Xi | Wen Wang | Chong Zhang | Xianghu Yue | ShiLiang Zhang | Haizhou Li
Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
The emergence of audio language models is empowered by neural audio codecs, which establish critical mappings between continuous waveforms and discrete tokens compatible with language model paradigms. The evolutionary trends from multi-layer residual vector quantizer to single-layer quantizer are beneficial for language-autoregressive decoding. However, the capability to handle multi-domain audio signals through a single codebook remains constrained by inter-domain distribution discrepancies. In this work, we introduce UniCodec, a unified audio codec with a single codebook to support multi-domain audio data, including speech, music, and sound. To achieve this, we propose a partitioned domain-adaptive codebook method based on domain Mixture-of-Experts strategy to capture the distinct characteristics of each audio domain. Furthermore, to enrich the semantic density of the codec without auxiliary modules, we propose a self-supervised mask prediction modeling approach. Comprehensive objective and subjective evaluations demonstrate that UniCodec achieves excellent audio reconstruction performance across the three audio domains, outperforming existing unified neural codecs with a single codebook, and even surpasses state-of-the-art domain-specific codecs on both acoustic and semantic representation capabilities.