The Bias Amplification Paradox in Text-to-Image Generation

Preethi Seshadri, Sameer Singh, Yanai Elazar


Abstract
Bias amplification is a phenomenon in which models exacerbate biases or stereotypes present in the training data. In this paper, we study bias amplification in the text-to-image domain using Stable Diffusion by comparing gender ratios in training vs. generated images. We find that the model appears to amplify gender-occupation biases found in the training data (LAION) considerably. However, we discover that amplification can be largely attributed to discrepancies between training captions and model prompts. For example, an inherent difference is that captions from the training data often contain explicit gender information while our prompts do not, which leads to a distribution shift and consequently inflates bias measures. Once we account for distributional differences between texts used for training and generation when evaluating amplification, we observe that amplification decreases drastically. Our findings illustrate the challenges of comparing biases in models and their training data, as well as evaluation more broadly, and highlight how confounding factors can impact analyses.
Anthology ID:
2024.naacl-long.353
Volume:
Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers)
Month:
June
Year:
2024
Address:
Mexico City, Mexico
Editors:
Kevin Duh, Helena Gomez, Steven Bethard
Venue:
NAACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
6367–6384
Language:
URL:
https://aclanthology.org/2024.naacl-long.353
DOI:
Bibkey:
Cite (ACL):
Preethi Seshadri, Sameer Singh, and Yanai Elazar. 2024. The Bias Amplification Paradox in Text-to-Image Generation. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), pages 6367–6384, Mexico City, Mexico. Association for Computational Linguistics.
Cite (Informal):
The Bias Amplification Paradox in Text-to-Image Generation (Seshadri et al., NAACL 2024)
Copy Citation:
PDF:
https://aclanthology.org/2024.naacl-long.353.pdf
Copyright:
 2024.naacl-long.353.copyright.pdf