Cellik Adams

Author directory

2026

Language models learn semantic similarity from shared context; however, when a single word has multiple orthographies that rarely appear in the same context, models cannot learn their equivalence. This is a structural problem that cannot be solved with scale. Korean presents a clear case: 20th-century print media used Hanja (Sino-Korean script), while contemporary models train on Hangul-only (Korean alphabet) text. Multilingual models inherit script segregation from their training distribution, where Hanja appears almost exclusively outside Korean text, and monolingual Korean models assign separate vocabulary entries to script variants, forcing distinct representations from initialization. We introduce Han2Han, a 169M parameter encoder-decoder model trained with a language-aware recipe—jamo and character-level embedding fusion, morpheme-aware denoising, and bidirectional Hanja-Hangul transcription—that forces orthographic alignment as a prior to distributional learning. The resulting representations are script-invariant: the same content receives the same representation regardless of script. On temporal classification of mixed-script historical newspapers, evaluated on an article-level held-out split, Han2Han reaches 76.0 macro F1 against 54.3 for the strongest baselines (KLUE-BERT and the 4.7× larger T5Gemma 2 alike), a 21.7-point margin, demonstrating that language-specific pre-training yields capabilities not addressed by current Korean-inclusive multilingual training at this scale.