Lifan Deng

Author directory

2026

Multilingual models are commonly described as supporting Chinese, but language-level coverage may conceal disparities between historical and modern varieties. We audit five public or widely deployed tokenizer families on Classical Chinese prose and Modern Chinese newspaper text. XLM-R, a canonical multilingual representation model, requires 1.070 tokens per Classical character versus 0.688 for Modern Chinese. More importantly, only 17.4% of annotated two-character Classical words receive a single XLM-R token, compared with 66.4% of Modern words. After controlling for word length and corpus frequency, the odds of atomic coverage are 0.158 of the Modern odds. Four general-purpose LLM tokenizers reproduce the same asymmetry. Exact sentence-length matching, frequency controls, Traditional-Simplified pairs, and leave-one-source-out tests rule out several simple artifacts. Multilingual tokenizers can therefore cover a language’s script while underrepresenting the lexical units of its historical varieties.
Classical Chinese poetry composition has long valued *Tuiqiao*, the iterative refinement of words, imagery, and prosody. However, many current AI poetry systems follow a one-shot generation paradigm, which reduces users to prompt providers and weakens their creative agency. We present *Jiuge-Tuiqiao*, an interactive human-AI collaborative system for classical Chinese poetry composition. The system is designed around a triadic model: user-driven control, ancient-guided evidence, and AI-assisted generation. Users can lock characters or lines, receive real-time prosody feedback, and obtain interpretable refinement suggestions grounded in high-frequency collocations, PPL-ranked classical lines, and structured knowledge extracted from classical encyclopedias. This design turns AI from an autonomous generator into a background assistant that supports the user’s own process of poetic refinement. Preliminary experiments and user feedback suggest that *Jiuge-Tuiqiao* improves controllability, interpretability, and user engagement in classical poetry composition.