Inspired by Large Language Models (LLMs), codec-based TTS systems have achieved remarkable success in high-fidelity speech generation by directly adopting the standard Cross-Entropy (CE) objective to predict sequences of discrete speech tokens from neural audio codecs. However, this practice overlooks the geometric structure of codec token spaces: tokens that are close in the codec codebook embedding space often correspond to acoustically similar speech realizations and can serve as perceptually plausible alternatives. We term this observation the Neighboring Token Phenomenon. The one-hot CE objective is agnostic to such structure, as it penalizes all non-target tokens equally, including these close neighbors, creating a mismatch between the structured codec latent space and the learning objective. In this paper, we quantitatively analyze the neighboring token phenomenon in both reconstruction and generation settings. To address the mismatch, we propose Multi-Neighbor Cross-Entropy (MNCE) loss function, which generalizes the one-hot target of CE loss into a positive set containing the ground-truth token and its nearest neighbors, encouraging the model to treat neighbor predictions as partially correct. As a result, even when the model misses the exact target token, it is more likely to predict a close neighbor rather than an unrelated token, reducing perceptually harmful errors in synthesized speech. Experiments demonstrate that MNCE is effective and generalizable across diverse codec-based TTS systems, improving quality and robustness. Moreover, scaling training to 20,000 hours further validates the scalability of our approach.

Building upon advancements in Large Language Models (LLMs), the field of audio processing has seen increased interest in training speech generation tasks with discrete speech token sequences. However, directly discretizing speech by neural audio codecs often results in sequences that fundamentally differ from text sequences. Unlike text, where text token sequences are deterministic, discrete speech tokens can exhibit significant variability based on contextual factors, while still producing perceptually identical audio segments. We refer to this phenomenon as Discrete Representation Inconsistency (DRI). This inconsistency can lead to a single speech segment being represented by multiple divergent sequences, which creates confusion in neural codec language models and results in poor generated speech. In this paper, we quantitatively analyze the DRI phenomenon within popular audio tokenizers such as EnCodec. Our approach effectively mitigates the DRI phenomenon of the neural audio codec. Furthermore, extensive experiments on the neural codec language model over LibriTTS and large-scale MLS dataset (44,000 hours) demonstrate the effectiveness and generality of our method. The demo of audio samples is available at https://consistencyinneuralcodec.github.io.

Philology, the study of ancient manuscripts, demands years of professional training in ex-tensive knowledge memorization and manual textual retrieval. Despite these requirements align closely with strengths of recent successful Large Language Models (LLMs), the scarcity of high-quality, specialized training data has hindered direct applications. To bridge this gap, we curated the PhiloCorpus-ZH, a rich collec-tion of ancient Chinese texts spanning a millen-nium with 30 diverse topics, including firsthand folk copies. This corpus facilitated the develop-ment of PhiloGPT, the first LLM tailored for discovering ancient Chinese manuscripts. To effectively tackle complex philological tasks like restoration, attribution, and linguistic anal-ysis, we introduced the PhiloCoP framework. Modeled on the analytical patterns of philol-ogists, PhiloCoP enhances LLM’s handling of historical linguistic peculiarities such as phonetic loans, polysemy, and syntactic inver-sions. We further integrated these tasks into the PhiloBenchmark, establishing a new standard for evaluating ancient Chinese LLMs address-ing philology tasks. Deploying PhiloGPT in practical scenarios has enabled Dunhuang spe-cialists to resolve philology tasks, such as iden-tifying duplication of copied text and assisting archaeologists with text completion, demon-strating its potential in real-world applications.