Recent work on gender bias in the automatic translation of gender-ambiguous scenarios has highlighted the need for gender ambiguity detection. We address this gap with ContraGAND, a contrastive corpus of 4,852 natural gender-ambiguous English sentences, each minimally paired with a masculine and a feminine variant generated by a neurosymbolic pipeline. We use ContraGAND to perform and evaluate gender ambiguity detection in two settings: (i) a closed task, where the referent is given, and (ii) an open task, where it is not. For both tasks, we evaluate six open-weight large language models by prompting, and test selected models with fine-tuning and knowledge distillation. We show that (i) the closed task is saturated in current models, whereas (ii) the open task produces a gap that varies considerably by model. We find that two failure modes dominate: a binding failure (the model surfaces the referent but defaults to ’ambiguous’ when the gender cue is cross-clause) and a format failure that, on local cues, is primarily under-commitment rather than a binding deficit (long unparseable reasoning chains on small models). Further analysis reveals that both are forms of over-prediction of the ’ambiguous’ class rather than stereotype-driven misclassification. While explicit prompts and decoding interventions do not close the gap, both fine-tuning on the augmented contrastive variants and distillation from a 31B teacher are effective at teaching a 4B student to successfully perform gender ambiguity detection.

As the demand for inclusive language increases, concern has grown over the susceptibility of machine translation (MT) systems to reinforce gender stereotypes. This study investigates gender bias in two commercial MT systems, Google Translate and DeepL, focusing on the understudied English-to-Greek language pair. We address three aspects of gender bias: i) male bias, ii) occupational stereotyping, and iii) errors in anti-stereotypical translations. Additionally, we explore the potential of prompted GPT-4o as a bias mitigation tool that provides both gender-explicit and gender-neutral alternatives when necessary. To achieve this, we introduce GendEL, a manually crafted bilingual dataset of 240 gender-ambiguous and unambiguous sentences that feature stereotypical occupational nouns and adjectives. We find persistent gender bias in translations by both MT systems; while they perform well in cases where gender is explicitly defined, with DeepL outperforming both Google Translate and GPT-4o in feminine gender-unambiguous sentences, they are far from producing gender-inclusive or neutral translations when the gender is unspecified. GPT-4o shows promise, generating appropriate gendered and neutral alternatives for most ambiguous cases, though residual biases remain evident. As one of the first comprehensive studies on gender bias in English-to-Greek MT, we provide both our data and code at https://github.com/elenigkove/genderbias_EN-EL_MT.

Contributing to research on gender beyond the binary, this work introduces GENDEROUS, a dataset of gender-ambiguous sentences containing gender-marked occupations and adjectives, and sentences with the ambiguous or non-binary pronoun their. We cross-linguistically evaluate how machine translation (MT) systems and large language models (LLMs) translate these sentences from English into four grammatical gender languages: Greek, German, Spanish and Dutch. We show the systems’ continued default to male-gendered translations, with exceptions (particularly for Dutch). Prompting for alternatives, however, shows potential in attaining more diverse and neutral translations across all languages. An LLM-as-a-judge approach was implemented, where benchmarking against gold standards emphasises the continued need for human annotations.

Avoiding the propagation of undue (binary) gender inferences and default masculine language remains a key challenge towards inclusive multilingual technologies, particularly when translating into languages with extensive gendered morphology. Gender-neutral translation (GNT) represents a linguistic strategy towards fairer communication across languages. However, research on GNT is limited to a few resources and language pairs. To address this gap, we introduce mGeNTE, an expert-curated resource, and use it to conduct the first systematic multilingual evaluation of inclusive translation with state-of-the-art instruction-following language models (LMs). Experiments on en-es/de/it/el reveal that while models can recognize when neutrality is appropriate, they cannot consistently produce neutral translations, limiting their usability. To probe this behavior, we enrich our evaluation with interpretability analyses that identify task-relevant features and offer initial insights into the internal dynamics of LM-based GNT.