Semi-supervised acoustic and language model training for English-isiZulu code-switched speech recognition

Astik Biswas, Febe De Wet, Ewald Van der westhuizen, Thomas Niesler


Abstract
We present an analysis of semi-supervised acoustic and language model training for English-isiZulu code-switched (CS) ASR using soap opera speech. Approximately 11 hours of untranscribed multilingual speech was transcribed automatically using four bilingual CS transcription systems operating in English-isiZulu, English-isiXhosa, English-Setswana and English-Sesotho. These transcriptions were incorporated into the acoustic and language model training sets. Results showed that the TDNN-F acoustic models benefit from the additional semi-supervised data and that even better performance could be achieved by including additional CNN layers. Using these CNN-TDNN-F acoustic models, a first iteration of semi-supervised training achieved an absolute mixed-language WER reduction of 3.44%, and a further 2.18% after a second iteration. Although the languages in the untranscribed data were unknown, the best results were obtained when all automatically transcribed data was used for training and not just the utterances classified as English-isiZulu. Despite perplexity improvements, the semi-supervised language model was not able to improve the ASR performance.
Anthology ID:
2020.calcs-1.7
Volume:
Proceedings of the 4th Workshop on Computational Approaches to Code Switching
Month:
May
Year:
2020
Address:
Marseille, France
Editors:
Thamar Solorio, Monojit Choudhury, Kalika Bali, Sunayana Sitaram, Amitava Das, Mona Diab
Venue:
CALCS
SIG:
Publisher:
European Language Resources Association
Note:
Pages:
52–56
Language:
English
URL:
https://aclanthology.org/2020.calcs-1.7
DOI:
Bibkey:
Cite (ACL):
Astik Biswas, Febe De Wet, Ewald Van der westhuizen, and Thomas Niesler. 2020. Semi-supervised acoustic and language model training for English-isiZulu code-switched speech recognition. In Proceedings of the 4th Workshop on Computational Approaches to Code Switching, pages 52–56, Marseille, France. European Language Resources Association.
Cite (Informal):
Semi-supervised acoustic and language model training for English-isiZulu code-switched speech recognition (Biswas et al., CALCS 2020)
Copy Citation:
PDF:
https://aclanthology.org/2020.calcs-1.7.pdf