Does Representational Fairness Imply Empirical Fairness?

Aili Shen, Xudong Han, Trevor Cohn, Timothy Baldwin, Lea Frermann


Abstract
NLP technologies can cause unintended harms if learned representations encode sensitive attributes of the author, or predictions systematically vary in quality across groups. Popular debiasing approaches, like adversarial training, remove sensitive information from representations in order to reduce disparate performance, however the relation between representational fairness and empirical (performance) fairness has not been systematically studied. This paper fills this gap, and proposes a novel debiasing method building on contrastive learning to encourage a latent space that separates instances based on target label, while mixing instances that share protected attributes. Our results show the effectiveness of our new method and, more importantly, show across a set of diverse debiasing methods that representational fairness does not imply empirical fairness. This work highlights the importance of aligning and understanding the relation of the optimization objective and final fairness target.
Anthology ID:
2022.findings-aacl.8
Volume:
Findings of the Association for Computational Linguistics: AACL-IJCNLP 2022
Month:
November
Year:
2022
Address:
Online only
Editors:
Yulan He, Heng Ji, Sujian Li, Yang Liu, Chua-Hui Chang
Venue:
Findings
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
81–95
Language:
URL:
https://aclanthology.org/2022.findings-aacl.8
DOI:
Bibkey:
Cite (ACL):
Aili Shen, Xudong Han, Trevor Cohn, Timothy Baldwin, and Lea Frermann. 2022. Does Representational Fairness Imply Empirical Fairness?. In Findings of the Association for Computational Linguistics: AACL-IJCNLP 2022, pages 81–95, Online only. Association for Computational Linguistics.
Cite (Informal):
Does Representational Fairness Imply Empirical Fairness? (Shen et al., Findings 2022)
Copy Citation:
PDF:
https://aclanthology.org/2022.findings-aacl.8.pdf