Gender bias amplification during Speed-Quality optimization in Neural Machine Translation

Adithya Renduchintala, Denise Diaz, Kenneth Heafield, Xian Li, Mona Diab


Abstract
Is bias amplified when neural machine translation (NMT) models are optimized for speed and evaluated on generic test sets using BLEU? We investigate architectures and techniques commonly used to speed up decoding in Transformer-based models, such as greedy search, quantization, average attention networks (AANs) and shallow decoder models and show their effect on gendered noun translation. We construct a new gender bias test set, SimpleGEN, based on gendered noun phrases in which there is a single, unambiguous, correct answer. While we find minimal overall BLEU degradation as we apply speed optimizations, we observe that gendered noun translation performance degrades at a much faster rate.
Anthology ID:
2021.acl-short.15
Volume:
Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 2: Short Papers)
Month:
August
Year:
2021
Address:
Online
Editors:
Chengqing Zong, Fei Xia, Wenjie Li, Roberto Navigli
Venues:
ACL | IJCNLP
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
99–109
Language:
URL:
https://aclanthology.org/2021.acl-short.15
DOI:
10.18653/v1/2021.acl-short.15
Bibkey:
Cite (ACL):
Adithya Renduchintala, Denise Diaz, Kenneth Heafield, Xian Li, and Mona Diab. 2021. Gender bias amplification during Speed-Quality optimization in Neural Machine Translation. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 2: Short Papers), pages 99–109, Online. Association for Computational Linguistics.
Cite (Informal):
Gender bias amplification during Speed-Quality optimization in Neural Machine Translation (Renduchintala et al., ACL-IJCNLP 2021)
Copy Citation:
PDF:
https://aclanthology.org/2021.acl-short.15.pdf
Optional supplementary material:
 2021.acl-short.15.OptionalSupplementaryMaterial.pdf
Video:
 https://aclanthology.org/2021.acl-short.15.mp4