THE IWSLT 2021 BUT SPEECH TRANSLATION SYSTEMS

hari Krishna Vydana, Martin Karafiat, Lukas Burget, Jan Černocký


Abstract
The paper describes BUT’s English to German offline speech translation (ST) systems developed for IWSLT2021. They are based on jointly trained Automatic Speech Recognition-Machine Translation models. Their performances is evaluated on MustC-Common test set. In this work, we study their efficiency from the perspective of having a large amount of separate ASR training data and MT training data, and a smaller amount of speech-translation training data. Large amounts of ASR and MT training data are utilized for pre-training the ASR and MT models. Speech-translation data is used to jointly optimize ASR-MT models by defining an end-to-end differentiable path from speech to translations. For this purpose, we use the internal continuous representations from the ASR-decoder as the input to MT module. We show that speech translation can be further improved by training the ASR-decoder jointly with the MT-module using large amount of text-only MT training data. We also show significant improvements by training an ASR module capable of generating punctuated text, rather than leaving the punctuation task to the MT module.
Anthology ID:
2021.iwslt-1.7
Volume:
Proceedings of the 18th International Conference on Spoken Language Translation (IWSLT 2021)
Month:
August
Year:
2021
Address:
Bangkok, Thailand (online)
Venues:
ACL | IJCNLP | IWSLT
SIG:
SIGSLT
Publisher:
Association for Computational Linguistics
Note:
Pages:
75–83
Language:
URL:
https://aclanthology.org/2021.iwslt-1.7
DOI:
10.18653/v1/2021.iwslt-1.7
Bibkey:
Cite (ACL):
hari Krishna Vydana, Martin Karafiat, Lukas Burget, and Jan Černocký. 2021. THE IWSLT 2021 BUT SPEECH TRANSLATION SYSTEMS. In Proceedings of the 18th International Conference on Spoken Language Translation (IWSLT 2021), pages 75–83, Bangkok, Thailand (online). Association for Computational Linguistics.
Cite (Informal):
THE IWSLT 2021 BUT SPEECH TRANSLATION SYSTEMS (Vydana et al., IWSLT 2021)
Copy Citation:
PDF:
https://aclanthology.org/2021.iwslt-1.7.pdf