A user study to compare two conversational assistants designed for people with hearing impairments

Anja Virkkunen, Juri Lukkarila, Kalle Palomäki, Mikko Kurimo


Abstract
Participating in conversations can be difficult for people with hearing loss, especially in acoustically challenging environments. We studied the preferences the hearing impaired have for a personal conversation assistant based on automatic speech recognition (ASR) technology. We created two prototypes which were evaluated by hearing impaired test users. This paper qualitatively compares the two based on the feedback obtained from the tests. The first prototype was a proof-of-concept system running real-time ASR on a laptop. The second prototype was developed for a mobile device with the recognizer running on a separate server. In the mobile device, augmented reality (AR) was used to help the hearing impaired observe gestures and lip movements of the speaker simultaneously with the transcriptions. Several testers found the systems useful enough to use in their daily lives, with majority preferring the mobile AR version. The biggest concern of the testers was the accuracy of the transcriptions and the lack of speaker identification.
Anthology ID:
W19-1701
Volume:
Proceedings of the Eighth Workshop on Speech and Language Processing for Assistive Technologies
Month:
June
Year:
2019
Address:
Minneapolis, Minnesota
Editors:
Heidi Christensen, Kristy Hollingshead, Emily Prud’hommeaux, Frank Rudzicz, Keith Vertanen
Venue:
SLPAT
SIG:
SIGSLPAT
Publisher:
Association for Computational Linguistics
Note:
Pages:
1–8
Language:
URL:
https://aclanthology.org/W19-1701
DOI:
10.18653/v1/W19-1701
Bibkey:
Cite (ACL):
Anja Virkkunen, Juri Lukkarila, Kalle Palomäki, and Mikko Kurimo. 2019. A user study to compare two conversational assistants designed for people with hearing impairments. In Proceedings of the Eighth Workshop on Speech and Language Processing for Assistive Technologies, pages 1–8, Minneapolis, Minnesota. Association for Computational Linguistics.
Cite (Informal):
A user study to compare two conversational assistants designed for people with hearing impairments (Virkkunen et al., SLPAT 2019)
Copy Citation:
PDF:
https://aclanthology.org/W19-1701.pdf