Carla Olabe
Author directory2026
Entropic properties of languages in phonetic versus orthographic alphabets
Carla Olabe | Christian Bentz
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Carla Olabe | Christian Bentz
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Computational linguistics predominantly operates on written text. However, writing represents spoken language only indirectly. Whether phonetic and orthographic representations share fundamental information-theoretic properties is an open question. We present a cross-linguistic analysis of Shannon entropy across modalities using the DoReCo corpus featuring 42 languages of 25 families after preprocessing. By computing continuous entropy trajectories at orthographic word, subword (BPE), and character levels (UTF-8/SAMPA), we reveal a systematic modality convergence: orthographic and phonetic encodings yield consistently small entropy differences across all three granularity levels. This general pattern, however, masks notable cross-linguistic variation–some languages, such as Evenki, converge tightly across all levels, while others show modality gaps at specific granularities–for instance, English at the character level, Bora at the word level, and Sanzhi Dargwa at the subword level. In sum, Shannon entropy tends toward modality-invariance as a general tendency across the Latin-script languages examined, while leaving room for typologically conditioned divergence. Whether this pattern extends to non-alphabetic writing systems remains an open question.