Better Hit the Nail on the Head than Beat around the Bush: Removing Protected Attributes with a Single Projection

Pantea Haghighatkhah, Antske Fokkens, Pia Sommerauer, Bettina Speckmann, Kevin Verbeek


Abstract
Bias elimination and recent probing studies attempt to remove specific information from embedding spaces. Here it is important to remove as much of the target information as possible, while preserving any other information present. INLP is a popular recent method which removes specific information through iterative nullspace projections. Multiple iterations, however, increase the risk that information other than the target is negatively affected. We introduce two methods that find a single targeted projection: Mean Projection (MP, more efficient) and Tukey Median Projection (TMP, with theoretical guarantees). Our comparison between MP and INLP shows that (1) one MP projection removes linear separability based on the target and (2) MP has less impact on the overall space. Further analysis shows that applying random projections after MP leads to the same overall effects on the embedding space as the multiple projections of INLP. Applying one targeted (MP) projection hence is methodologically cleaner than applying multiple (INLP) projections that introduce random effects.
Anthology ID:
2022.emnlp-main.575
Volume:
Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing
Month:
December
Year:
2022
Address:
Abu Dhabi, United Arab Emirates
Editors:
Yoav Goldberg, Zornitsa Kozareva, Yue Zhang
Venue:
EMNLP
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
8395–8416
Language:
URL:
https://aclanthology.org/2022.emnlp-main.575
DOI:
10.18653/v1/2022.emnlp-main.575
Bibkey:
Cite (ACL):
Pantea Haghighatkhah, Antske Fokkens, Pia Sommerauer, Bettina Speckmann, and Kevin Verbeek. 2022. Better Hit the Nail on the Head than Beat around the Bush: Removing Protected Attributes with a Single Projection. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 8395–8416, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics.
Cite (Informal):
Better Hit the Nail on the Head than Beat around the Bush: Removing Protected Attributes with a Single Projection (Haghighatkhah et al., EMNLP 2022)
Copy Citation:
PDF:
https://aclanthology.org/2022.emnlp-main.575.pdf