Yusuf Öztop

Author directory

2026

Turkish has no grammatical gender: its third-person pronoun o refers to a man, a woman, or an inanimate object alike, so a system translating into English must supply a gender the source never states. We introduce tr_gender_bias, a reference-free Turkish-to-English test suite of 200 templated items, each built around a single human referent, and use it to measure how 24 machine translation systems and large language models assign gender when the source withholds it. The suite has three parts: ambiguous items whose only signal is an occupational stereotype grounded in labor-force statistics, items whose gender is fixed by a kinship term or a name against that stereotype, and neutral controls. Every output is scored under a six-way, abstention-aware taxonomy that treats neutralization, hedging, and malformed output as distinct outcomes. Under ambiguity, stereotype-driven gender assignment is universal, significant for all 22 testable systems after correction, and uniformly male-skewed, with no system ever favoring the feminine. The effect is sharply asymmetric: every system renders male-typed occupations as he almost without exception, while agreement on female-typed occupations is barely above chance, so what separates systems is how strong a feminine stereotype must be to displace the masculine default. Gender that the context states, by contrast, is now resolved almost perfectly, a marked change from earlier evaluations in which stereotype overrode context. The systems split into committers and neutralizers, and neutralizing with singular they removes both stereotype bias and male skew at no cost in accuracy. Gender bias in current translation is therefore less a failure to use context than an unsettled policy for what to do when the source leaves gender genuinely open.
This paper presents the KYX submission to the WMT 2026 CreoleMT shared task for Réunion Creole to English (rcf-eng). It is mostly about how such a system should be evaluated. The pair is an extreme case: 191 training pairs, a 34-sentence development set, and no standard orthography. On a benchmark this small, a single chrF++ score is easy to over-read, so we pair our system with a contamination- and orthography-aware evaluation. The system continue-trains the official baseline adapter with QLoRA and raises the raw dev score by +13.3 chrF++. But a near-duplicate audit finds that six of the 34 dev sources are paraphrases of training sentences, which the exact-match check WMT26 mandates reports as 0% overlap. After decontamination the gain is +10.0. Our official test score then comes in 0.8 chrF++ below the decontaminated estimate and 5.1 below the raw one, but our margin over the baseline falls to +3.9, so even the adjusted gain was optimistic. A data-scaling ablation traces the gain to the provided real pairs, not to synthetic data. A source-spelling stress test then shows the advantage shrinks when the input is re-spelled into equally valid variants, though at this sample size the effect is not stable. We conclude that exact-match contamination checks are not enough on tiny benchmarks, and that scores like ours should be read with care. The same caution applies to any low-resource pair with only a few dozen dev sentences. We release the system, the audit, and all data.
Search
Co-authors
    Venues
    Fix author