Contaminated Collaboration: Measuring Gender Bias Transfer in LLM-Assisted Student Writing

Ariyan Hossain, Kazi Kamruzzaman Rabbi, Farig Sadeque, S M Taiabul Haque


Abstract
Gender bias in LLMs has been studied extensively in model outputs, with biased prompts shown to amplify stereotyped generations. Whether such bias propagates into text produced by humans who use these systems, however, remains underexplored. We investigate whether gender bias in an LLM writing assistant transfers into career plan essays written by students. We first verify that a gender-biased prompt induces gender-differentiated language in LLM-generated essays, while a neutral prompt does not. We then recruited participants (N = 123) in a controlled environment to write career plan essays for paired biographical profiles differing only in gender under three conditions: no AI assistance, neutral LLM assistance, or gender-biased LLM assistance. Students in the biased condition produced essays with a significantly larger agentic gap and more gender-stereotypic occupation suggestions than those in the control and neutral conditions. Our results also reveal that this bias transfer is asymmetric: agency is suppressed in female-target essays while male-target writing remains largely unaffected. Our findings highlight the risk of bias propagation in AI-assisted writing, calling for fairness-aware design in educational AI tools.
Anthology ID:
2026.emnlp-main.1481
Volume:
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Month:
October
Year:
2026
Address:
Budapest, Hungary
Editors:
Hung-Yi Lee, Sunipa Dev, Jessy Li, Desmond Elliott
Venue:
EMNLP
SIG:
SIGDAT
Publisher:
Association for Computational Linguistics
Note:
Pages:
33846–33864
Language:
URL:
https://aclanthology.org/2026.emnlp-main.1481/
DOI:
Bibkey:
Cite (ACL):
Ariyan Hossain, Kazi Kamruzzaman Rabbi, Farig Sadeque, and S M Taiabul Haque. 2026. Contaminated Collaboration: Measuring Gender Bias Transfer in LLM-Assisted Student Writing. In Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing, pages 33846–33864, Budapest, Hungary. Association for Computational Linguistics.
Cite (Informal):
Contaminated Collaboration: Measuring Gender Bias Transfer in LLM-Assisted Student Writing (Hossain et al., EMNLP 2026)
Copy Citation:
PDF:
https://aclanthology.org/2026.emnlp-main.1481.pdf