Exploring the Integration of E2E ASR and Pronunciation Modeling for English Mispronunciation Detection

Hsin-Wei Wang, Bi-Cheng Yan, Yung-Chang Hsu, Berlin Chen


Abstract
There has been increasing demand to develop effective computer-assisted language training (CAPT) systems, which can provide feedback on mispronunciations and facilitate second-language (L2) learners to improve their speaking proficiency through repeated practice. Due to the shortage of non-native speech for training the automatic speech recognition (ASR) module of a CAPT system, the corresponding mispronunciation detection performance is often affected by imperfect ASR. Recognizing this importance, we in this paper put forward a two-stage mispronunciation detection method. In the first stage, the speech uttered by an L2 learner is processed by an end-to-end ASR module to produce N-best phone sequence hypotheses. In the second stage, these hypotheses are fed into a pronunciation model which seeks to faithfully predict the phone sequence hypothesis that is most likely pronounced by the learner, so as to improve the performance of mispronunciation detection. Empirical experiments conducted a English benchmark dataset seem to confirm the utility of our method.
Anthology ID:
2021.rocling-1.17
Volume:
Proceedings of the 33rd Conference on Computational Linguistics and Speech Processing (ROCLING 2021)
Month:
October
Year:
2021
Address:
Taoyuan, Taiwan
Venue:
ROCLING
SIG:
Publisher:
The Association for Computational Linguistics and Chinese Language Processing (ACLCLP)
Note:
Pages:
124–131
Language:
URL:
https://aclanthology.org/2021.rocling-1.17
DOI:
Bibkey:
Cite (ACL):
Hsin-Wei Wang, Bi-Cheng Yan, Yung-Chang Hsu, and Berlin Chen. 2021. Exploring the Integration of E2E ASR and Pronunciation Modeling for English Mispronunciation Detection. In Proceedings of the 33rd Conference on Computational Linguistics and Speech Processing (ROCLING 2021), pages 124–131, Taoyuan, Taiwan. The Association for Computational Linguistics and Chinese Language Processing (ACLCLP).
Cite (Informal):
Exploring the Integration of E2E ASR and Pronunciation Modeling for English Mispronunciation Detection (Wang et al., ROCLING 2021)
Copy Citation:
PDF:
https://aclanthology.org/2021.rocling-1.17.pdf