Learning Fair Representations via Rate-Distortion Maximization

Somnath Basu Roy Chowdhury, Snigdha Chaturvedi


Abstract
Text representations learned by machine learning models often encode undesirable demographic information of the user. Predictive models based on these representations can rely on such information, resulting in biased decisions. We present a novel debiasing technique, Fairness-aware Rate Maximization (FaRM), that removes protected information by making representations of instances belonging to the same protected attribute class uncorrelated, using the rate-distortion function. FaRM is able to debias representations with or without a target task at hand. FaRM can also be adapted to remove information about multiple protected attributes simultaneously. Empirical evaluations show that FaRM achieves state-of-the-art performance on several datasets, and learned representations leak significantly less protected attribute information against an attack by a non-linear probing network.
Anthology ID:
2022.tacl-1.67
Volume:
Transactions of the Association for Computational Linguistics, Volume 10
Month:
Year:
2022
Address:
Cambridge, MA
Editors:
Brian Roark, Ani Nenkova
Venue:
TACL
SIG:
Publisher:
MIT Press
Note:
Pages:
1159–1174
Language:
URL:
https://aclanthology.org/2022.tacl-1.67
DOI:
10.1162/tacl_a_00512
Bibkey:
Cite (ACL):
Somnath Basu Roy Chowdhury and Snigdha Chaturvedi. 2022. Learning Fair Representations via Rate-Distortion Maximization. Transactions of the Association for Computational Linguistics, 10:1159–1174.
Cite (Informal):
Learning Fair Representations via Rate-Distortion Maximization (Chowdhury & Chaturvedi, TACL 2022)
Copy Citation:
PDF:
https://aclanthology.org/2022.tacl-1.67.pdf
Video:
 https://aclanthology.org/2022.tacl-1.67.mp4