Context-sensitive evaluation of automatic speech recognition: considering user experience & language variation

Nina Markl, Catherine Lai


Abstract
Commercial Automatic Speech Recognition (ASR) systems tend to show systemic predictive bias for marginalised speaker/user groups. We highlight the need for an interdisciplinary and context-sensitive approach to documenting this bias incorporating perspectives and methods from sociolinguistics, speech & language technology and human-computer interaction in the context of a case study. We argue evaluation of ASR systems should be disaggregated by speaker group, include qualitative error analysis, and consider user experience in a broader sociolinguistic and social context.
Anthology ID:
2021.hcinlp-1.6
Volume:
Proceedings of the First Workshop on Bridging Human–Computer Interaction and Natural Language Processing
Month:
April
Year:
2021
Address:
Online
Venues:
EACL | HCINLP
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
34–40
Language:
URL:
https://aclanthology.org/2021.hcinlp-1.6
DOI:
Bibkey:
Cite (ACL):
Nina Markl and Catherine Lai. 2021. Context-sensitive evaluation of automatic speech recognition: considering user experience & language variation. In Proceedings of the First Workshop on Bridging Human–Computer Interaction and Natural Language Processing, pages 34–40, Online. Association for Computational Linguistics.
Cite (Informal):
Context-sensitive evaluation of automatic speech recognition: considering user experience & language variation (Markl & Lai, HCINLP 2021)
Copy Citation:
PDF:
https://aclanthology.org/2021.hcinlp-1.6.pdf