Dialects of Translationese Shape Language Model Learning

Jenny Kunz


Abstract
Machine-translated data is widely used in multilingual NLP, particularly where native text is scarce. However, translated text differs systematically from native text. This phenomenon is known as translationese, and it reflects both traces of the source language and characteristic properties of translation itself. In this paper, we study how training on machine-translated data affects small English language models, focusing on how translationese from different source languages shapes linguistic acceptability judgments and language modeling for different domains. We train models on English text translated from 24 typologically and resource-diverse source languages, enabling a systematic analysis of how source language and corpus properties influence what models learn. Our results show that the source language has a clear impact on model behavior: general perplexity is more driven by the lexical diversity of the translated corpus, but grammatical performance is strongly correlated to typological similarity to English if trained on enough data. Even translation quality is a strong predictor of language modeling performance.
Anthology ID:
2026.findings-emnlp.2272
Volume:
Findings of the Association for Computational Linguistics: EMNLP 2026
Month:
October
Year:
2026
Address:
Budapest, Hungary
Editors:
Hung-Yi Lee, Sunipa Dev, Jessy Li, Desmond Elliott
Venue:
Findings
SIG:
SIGDAT
Publisher:
Association for Computational Linguistics
Note:
Pages:
48640–48659
Language:
URL:
https://aclanthology.org/2026.findings-emnlp.2272/
DOI:
Bibkey:
Cite (ACL):
Jenny Kunz. 2026. Dialects of Translationese Shape Language Model Learning. In Findings of the Association for Computational Linguistics: EMNLP 2026, pages 48640–48659, Budapest, Hungary. Association for Computational Linguistics.
Cite (Informal):
Dialects of Translationese Shape Language Model Learning (Kunz, Findings 2026)
Copy Citation:
PDF:
https://aclanthology.org/2026.findings-emnlp.2272.pdf