Exploring the Linear Subspace Hypothesis in Gender Bias Mitigation

Francisco Vargas, Ryan Cotterell


Abstract
Bolukbasi et al. (2016) presents one of the first gender bias mitigation techniques for word embeddings. Their method takes pre-trained word embeddings as input and attempts to isolate a linear subspace that captures most of the gender bias in the embeddings. As judged by an analogical evaluation task, their method virtually eliminates gender bias in the embeddings. However, an implicit and untested assumption of their method is that the bias subspace is actually linear. In this work, we generalize their method to a kernelized, non-linear version. We take inspiration from kernel principal component analysis and derive a non-linear bias isolation technique. We discuss and overcome some of the practical drawbacks of our method for non-linear gender bias mitigation in word embeddings and analyze empirically whether the bias subspace is actually linear. Our analysis shows that gender bias is in fact well captured by a linear subspace, justifying the assumption of Bolukbasi et al. (2016).
Anthology ID:
2020.emnlp-main.232
Volume:
Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)
Month:
November
Year:
2020
Address:
Online
Editors:
Bonnie Webber, Trevor Cohn, Yulan He, Yang Liu
Venue:
EMNLP
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
2902–2913
Language:
URL:
https://aclanthology.org/2020.emnlp-main.232
DOI:
10.18653/v1/2020.emnlp-main.232
Bibkey:
Cite (ACL):
Francisco Vargas and Ryan Cotterell. 2020. Exploring the Linear Subspace Hypothesis in Gender Bias Mitigation. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 2902–2913, Online. Association for Computational Linguistics.
Cite (Informal):
Exploring the Linear Subspace Hypothesis in Gender Bias Mitigation (Vargas & Cotterell, EMNLP 2020)
Copy Citation:
PDF:
https://aclanthology.org/2020.emnlp-main.232.pdf
Optional supplementary material:
 2020.emnlp-main.232.OptionalSupplementaryMaterial.zip
Video:
 https://slideslive.com/38938828
Code
 franciscovargas/Bias_space_study