Language Modeling Across Script Variants: The Case of Japanese

Go Inoue, Nizar Habash, Timothy Baldwin


Abstract
We investigate the effect of script variants on language modeling in Japanese, which uniquely combines logographic (kanji) and phonetic (kana) writing systems. Motivated by educational curricula in which learners are gradually exposed to increasing numbers of logographic characters, we simulate this staged exposure during training to study its effect on model behavior. We construct multiple versions of a corpus with controlled script transformations, including fully transliterated kana text and intermediate variants corresponding to curriculum-based kanji levels, and train language models under different schedules that vary the timing and progression of script exposure during pre-training. In addition, we propose a cross-script evaluation framework using multiple versions of a minimal-pair benchmark with systematically varied kanji usage to assess the robustness of language models to script variation. Our results show that training on a mixture of two extreme script variants improves robustness across script conditions, whereas curriculum-based progression does not consistently improve performance.
Anthology ID:
2026.findings-emnlp.2151
Volume:
Findings of the Association for Computational Linguistics: EMNLP 2026
Month:
October
Year:
2026
Address:
Budapest, Hungary
Editors:
Hung-Yi Lee, Sunipa Dev, Jessy Li, Desmond Elliott
Venue:
Findings
SIG:
SIGDAT
Publisher:
Association for Computational Linguistics
Note:
Pages:
45964–45980
Language:
URL:
https://aclanthology.org/2026.findings-emnlp.2151/
DOI:
Bibkey:
Cite (ACL):
Go Inoue, Nizar Habash, and Timothy Baldwin. 2026. Language Modeling Across Script Variants: The Case of Japanese. In Findings of the Association for Computational Linguistics: EMNLP 2026, pages 45964–45980, Budapest, Hungary. Association for Computational Linguistics.
Cite (Informal):
Language Modeling Across Script Variants: The Case of Japanese (Inoue et al., Findings 2026)
Copy Citation:
PDF:
https://aclanthology.org/2026.findings-emnlp.2151.pdf