ContraGAND: Auditing and Repairing Gender Ambiguity Failures in LLMs with Neurosymbolic Contrastive Data Augmentation

Thomas Moerman, Eleni Gkovedarou, Janiça Hackenbuchner


Abstract
Recent work on gender bias in the automatic translation of gender-ambiguous scenarios has highlighted the need for gender ambiguity detection. We address this gap with ContraGAND, a contrastive corpus of 4,852 natural gender-ambiguous English sentences, each minimally paired with a masculine and a feminine variant generated by a neurosymbolic pipeline. We use ContraGAND to perform and evaluate gender ambiguity detection in two settings: (i) a closed task, where the referent is given, and (ii) an open task, where it is not. For both tasks, we evaluate six open-weight large language models by prompting, and test selected models with fine-tuning and knowledge distillation. We show that (i) the closed task is saturated in current models, whereas (ii) the open task produces a gap that varies considerably by model. We find that two failure modes dominate: a binding failure (the model surfaces the referent but defaults to ’ambiguous’ when the gender cue is cross-clause) and a format failure that, on local cues, is primarily under-commitment rather than a binding deficit (long unparseable reasoning chains on small models). Further analysis reveals that both are forms of over-prediction of the ’ambiguous’ class rather than stereotype-driven misclassification. While explicit prompts and decoding interventions do not close the gap, both fine-tuning on the augmented contrastive variants and distillation from a 31B teacher are effective at teaching a 4B student to successfully perform gender ambiguity detection.
Anthology ID:
2026.emnlp-main.485
Volume:
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Month:
October
Year:
2026
Address:
Budapest, Hungary
Editors:
Hung-Yi Lee, Sunipa Dev, Jessy Li, Desmond Elliott
Venue:
EMNLP
SIG:
SIGDAT
Publisher:
Association for Computational Linguistics
Note:
Pages:
11136–11159
Language:
URL:
https://aclanthology.org/2026.emnlp-main.485/
DOI:
Bibkey:
Cite (ACL):
Thomas Moerman, Eleni Gkovedarou, and Janiça Hackenbuchner. 2026. ContraGAND: Auditing and Repairing Gender Ambiguity Failures in LLMs with Neurosymbolic Contrastive Data Augmentation. In Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing, pages 11136–11159, Budapest, Hungary. Association for Computational Linguistics.
Cite (Informal):
ContraGAND: Auditing and Repairing Gender Ambiguity Failures in LLMs with Neurosymbolic Contrastive Data Augmentation (Moerman et al., EMNLP 2026)
Copy Citation:
PDF:
https://aclanthology.org/2026.emnlp-main.485.pdf