On the Impact of Random Seeds on the Fairness of Clinical Classifiers

Silvio Amir, Jan-Willem van de Meent, Byron Wallace


Abstract
Recent work has shown that fine-tuning large networks is surprisingly sensitive to changes in random seed(s). We explore the implications of this phenomenon for model fairness across demographic groups in clinical prediction tasks over electronic health records (EHR) in MIMIC-III —— the standard dataset in clinical NLP research. Apparent subgroup performance varies substantially for seeds that yield similar overall performance, although there is no evidence of a trade-off between overall and subgroup performance. However, we also find that the small sample sizes inherent to looking at intersections of minority groups and somewhat rare conditions limit our ability to accurately estimate disparities. Further, we find that jointly optimizing for high overall performance and low disparities does not yield statistically significant improvements. Our results suggest that fairness work using MIMIC-III should carefully account for variations in apparent differences that may arise from stochasticity and small sample sizes.
Anthology ID:
2021.naacl-main.299
Volume:
Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies
Month:
June
Year:
2021
Address:
Online
Editors:
Kristina Toutanova, Anna Rumshisky, Luke Zettlemoyer, Dilek Hakkani-Tur, Iz Beltagy, Steven Bethard, Ryan Cotterell, Tanmoy Chakraborty, Yichao Zhou
Venue:
NAACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
3808–3823
Language:
URL:
https://aclanthology.org/2021.naacl-main.299
DOI:
10.18653/v1/2021.naacl-main.299
Bibkey:
Cite (ACL):
Silvio Amir, Jan-Willem van de Meent, and Byron Wallace. 2021. On the Impact of Random Seeds on the Fairness of Clinical Classifiers. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 3808–3823, Online. Association for Computational Linguistics.
Cite (Informal):
On the Impact of Random Seeds on the Fairness of Clinical Classifiers (Amir et al., NAACL 2021)
Copy Citation:
PDF:
https://aclanthology.org/2021.naacl-main.299.pdf
Video:
 https://aclanthology.org/2021.naacl-main.299.mp4