Han2Han: Efficient Language-Specific Character Representation through Script-Aware Pre-Training for Historical Text Analysis

Cellik Adams, EunKyoung Jo, Juae Kim


Abstract
Language models learn semantic similarity from shared context; however, when a single word has multiple orthographies that rarely appear in the same context, models cannot learn their equivalence. This is a structural problem that cannot be solved with scale. Korean presents a clear case: 20th-century print media used Hanja (Sino-Korean script), while contemporary models train on Hangul-only (Korean alphabet) text. Multilingual models inherit script segregation from their training distribution, where Hanja appears almost exclusively outside Korean text, and monolingual Korean models assign separate vocabulary entries to script variants, forcing distinct representations from initialization. We introduce Han2Han, a 169M parameter encoder-decoder model trained with a language-aware recipe—jamo and character-level embedding fusion, morpheme-aware denoising, and bidirectional Hanja-Hangul transcription—that forces orthographic alignment as a prior to distributional learning. The resulting representations are script-invariant: the same content receives the same representation regardless of script. On temporal classification of mixed-script historical newspapers, evaluated on an article-level held-out split, Han2Han reaches 76.0 macro F1 against 54.3 for the strongest baselines (KLUE-BERT and the 4.7× larger T5Gemma 2 alike), a 21.7-point margin, demonstrating that language-specific pre-training yields capabilities not addressed by current Korean-inclusive multilingual training at this scale.
Anthology ID:
2026.findings-emnlp.2146
Volume:
Findings of the Association for Computational Linguistics: EMNLP 2026
Month:
October
Year:
2026
Address:
Budapest, Hungary
Editors:
Hung-Yi Lee, Sunipa Dev, Jessy Li, Desmond Elliott
Venue:
Findings
SIG:
SIGDAT
Publisher:
Association for Computational Linguistics
Note:
Pages:
45857–45876
Language:
URL:
https://aclanthology.org/2026.findings-emnlp.2146/
DOI:
Bibkey:
Cite (ACL):
Cellik Adams, EunKyoung Jo, and Juae Kim. 2026. Han2Han: Efficient Language-Specific Character Representation through Script-Aware Pre-Training for Historical Text Analysis. In Findings of the Association for Computational Linguistics: EMNLP 2026, pages 45857–45876, Budapest, Hungary. Association for Computational Linguistics.
Cite (Informal):
Han2Han: Efficient Language-Specific Character Representation through Script-Aware Pre-Training for Historical Text Analysis (Adams et al., Findings 2026)
Copy Citation:
PDF:
https://aclanthology.org/2026.findings-emnlp.2146.pdf