Anna Zee


2024

pdf bib
Group Fairness in Multilingual Speech Recognition Models
Anna Zee | Marc Zee | Anders Søgaard
Findings of the Association for Computational Linguistics: NAACL 2024

We evaluate the performance disparity of the Whisper and MMS families of ASR models across the VoxPopuli and Common Voice multilingual datasets, with an eye toward intersectionality. Our two most important findings are that model size, surprisingly, correlates logarithmically with worst-case performance disparities, meaning that larger (and better) models are less fair. We also observe the importance of intersectionality. In particular, models often exhibit significant performance disparity across binary gender for adolescents.