Transferring Fairness using Multi-Task Learning with Limited Demographic Information

Carlos Alejandro Aguirre, Mark Dredze


Abstract
Training supervised machine learning systems with a fairness loss can improve prediction fairness across different demographic groups. However, doing so requires demographic annotations for training data, without which we cannot produce debiased classifiers for most tasks. Drawing inspiration from transfer learning methods, we investigate whether we can utilize demographic data from a related task to improve the fairness of a target task. We adapt a single-task fairness loss to a multi-task setting to exploit demographic labels from a related task in debiasing a target task, and demonstrate that demographic fairness objectives transfer fairness within a multi-task framework. Additionally, we show that this approach enables intersectional fairness by transferring between two datasets with different single-axis demographics. We explore different data domains to show how our loss can improve fairness domains and tasks.
Anthology ID:
2024.nlp4pi-1.3
Volume:
Proceedings of the Third Workshop on NLP for Positive Impact
Month:
November
Year:
2024
Address:
Miami, Florida, USA
Editors:
Daryna Dementieva, Oana Ignat, Zhijing Jin, Rada Mihalcea, Giorgio Piatti, Joel Tetreault, Steven Wilson, Jieyu Zhao
Venue:
NLP4PI
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
32–49
Language:
URL:
https://aclanthology.org/2024.nlp4pi-1.3
DOI:
Bibkey:
Cite (ACL):
Carlos Alejandro Aguirre and Mark Dredze. 2024. Transferring Fairness using Multi-Task Learning with Limited Demographic Information. In Proceedings of the Third Workshop on NLP for Positive Impact, pages 32–49, Miami, Florida, USA. Association for Computational Linguistics.
Cite (Informal):
Transferring Fairness using Multi-Task Learning with Limited Demographic Information (Aguirre & Dredze, NLP4PI 2024)
Copy Citation:
PDF:
https://aclanthology.org/2024.nlp4pi-1.3.pdf