Systematic Inequalities in Language Technology Performance across the World’s Languages

Damian Blasi, Antonios Anastasopoulos, Graham Neubig


Abstract
Natural language processing (NLP) systems have become a central technology in communication, education, medicine, artificial intelligence, and many other domains of research and development. While the performance of NLP methods has grown enormously over the last decade, this progress has been restricted to a minuscule subset of the world’s 6,500 languages. We introduce a framework for estimating the global utility of language technologies as revealed in a comprehensive snapshot of recent publications in NLP. Our analyses involve the field at large, but also more in-depth studies on both user-facing technologies (machine translation, language understanding, question answering, text-to-speech synthesis) as well as foundational NLP tasks (dependency parsing, morphological inflection). In the process, we (1) quantify disparities in the current state of NLP research, (2) explore some of its associated societal and academic factors, and (3) produce tailored recommendations for evidence-based policy making aimed at promoting more global and equitable language technologies. Data and code to reproduce the findings discussed in this paper areavailable on GitHub (https://github.com/neubig/globalutility).
Anthology ID:
2022.acl-long.376
Volume:
Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
Month:
May
Year:
2022
Address:
Dublin, Ireland
Editors:
Smaranda Muresan, Preslav Nakov, Aline Villavicencio
Venue:
ACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
5486–5505
Language:
URL:
https://aclanthology.org/2022.acl-long.376
DOI:
10.18653/v1/2022.acl-long.376
Bibkey:
Cite (ACL):
Damian Blasi, Antonios Anastasopoulos, and Graham Neubig. 2022. Systematic Inequalities in Language Technology Performance across the World’s Languages. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 5486–5505, Dublin, Ireland. Association for Computational Linguistics.
Cite (Informal):
Systematic Inequalities in Language Technology Performance across the World’s Languages (Blasi et al., ACL 2022)
Copy Citation:
PDF:
https://aclanthology.org/2022.acl-long.376.pdf
Software:
 2022.acl-long.376.software.zip
Video:
 https://aclanthology.org/2022.acl-long.376.mp4
Code
 neubig/globalutility