Evaluating Occupational Gender Bias and Male Default in Turkish-to-English Machine Translation

Yusuf Öztop


Abstract
Turkish has no grammatical gender: its third-person pronoun o refers to a man, a woman, or an inanimate object alike, so a system translating into English must supply a gender the source never states. We introduce tr_gender_bias, a reference-free Turkish-to-English test suite of 200 templated items, each built around a single human referent, and use it to measure how 24 machine translation systems and large language models assign gender when the source withholds it. The suite has three parts: ambiguous items whose only signal is an occupational stereotype grounded in labor-force statistics, items whose gender is fixed by a kinship term or a name against that stereotype, and neutral controls. Every output is scored under a six-way, abstention-aware taxonomy that treats neutralization, hedging, and malformed output as distinct outcomes. Under ambiguity, stereotype-driven gender assignment is universal, significant for all 22 testable systems after correction, and uniformly male-skewed, with no system ever favoring the feminine. The effect is sharply asymmetric: every system renders male-typed occupations as he almost without exception, while agreement on female-typed occupations is barely above chance, so what separates systems is how strong a feminine stereotype must be to displace the masculine default. Gender that the context states, by contrast, is now resolved almost perfectly, a marked change from earlier evaluations in which stereotype overrode context. The systems split into committers and neutralizers, and neutralizing with singular they removes both stereotype bias and male skew at no cost in accuracy. Gender bias in current translation is therefore less a failure to use context than an unsettled policy for what to do when the source leaves gender genuinely open.
Anthology ID:
2026.wmt-main.90
Volume:
Proceedings of the Eleventh Conference on Machine Translation
Month:
October
Year:
2026
Address:
Budapest, Hungary
Editors:
Barry Haddow, Tom Kocmi, Philipp Koehn, Christof Monz
Venues:
WMT | WS
SIG:
SIGMT
Publisher:
Association for Computational Linguistics
Note:
Pages:
1568–1576
Language:
URL:
https://aclanthology.org/2026.wmt-main.90/
DOI:
Bibkey:
Cite (ACL):
Yusuf Öztop. 2026. Evaluating Occupational Gender Bias and Male Default in Turkish-to-English Machine Translation. In Proceedings of the Eleventh Conference on Machine Translation, pages 1568–1576, Budapest, Hungary. Association for Computational Linguistics.
Cite (Informal):
Evaluating Occupational Gender Bias and Male Default in Turkish-to-English Machine Translation (Öztop, WMT 2026)
Copy Citation:
PDF:
https://aclanthology.org/2026.wmt-main.90.pdf