Analyzing Occupational Gender Bias across the LLM Training Lifecycle

Manthan Gadhia, Alexander Hoyle, Karolina Stańczak


Abstract
Post-training is widely assumed to reduce the social biases language models inherit from pre-training, yet direct evidence for this assumption remains mixed. In this paper, we examine how gender disparities evolve across the language modeling pipeline. In particular, we study occupational gender bias in OLMo 3, a fully open language model family, across its post-training stages by prompting each checkpoint to generate open-ended narratives about synthetic personas given their occupation and education. We consider two settings, a gender-given setting in which the profile states the persona’s gender explicitly and a gender-assumed setting in which it must be inferred by the model. We find that when gender is stated, post-training increases semantic diversity for male-coded profiles, whereas for female-coded profiles the increase is smaller after SFT and not significant after DPO and RLVR. When gender is assumed by the model, post-training affects narratives about female and male personas similarly. The model’s own gender assumptions, however, show a consistent female-preferential skew relative to United States labor-force demographics, except for heavily male-dominated occupations. Every successive post-training stage monotonically increases the odds of attributing a female label over the base model. A provenance analysis of the supervised fine-tuning corpus finds only a modest 7.7% female-token surplus, too small to explain the size of these downstream effects. The project code is available under: https://github.com/manthangadhia/llm-lifecycle-bias.
Anthology ID:
2026.gebnlp-1.11
Volume:
Proceedings of the 7th Workshop on Gender Bias in Natural Language Processing (GeBNLP)
Month:
November
Year:
2026
Address:
Hengqin, China (remote)
Editors:
Agnieszka Faleńska, Christine Basta, Marta Costa-jussà, Karolina Stańczak, Debora Nozza, Giuseppe Attanasio, Vera Neplenbroek, Christian Hardmeier
Venues:
GeBNLP | WS
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
157–172
Language:
URL:
https://aclanthology.org/2026.gebnlp-1.11/
DOI:
Bibkey:
Cite (ACL):
Manthan Gadhia, Alexander Hoyle, and Karolina Stańczak. 2026. Analyzing Occupational Gender Bias across the LLM Training Lifecycle. In Proceedings of the 7th Workshop on Gender Bias in Natural Language Processing (GeBNLP), pages 157–172, Hengqin, China (remote). Association for Computational Linguistics.
Cite (Informal):
Analyzing Occupational Gender Bias across the LLM Training Lifecycle (Gadhia et al., GeBNLP 2026)
Copy Citation:
PDF:
https://aclanthology.org/2026.gebnlp-1.11.pdf
Sources:
 2026.gebnlp-1.11.sources.zip