LibriS2S: A German-English Speech-to-Speech Translation Corpus

Pedro Jeuris, Jan Niehues


Abstract
Recently, we have seen an increasing interest in the area of speech-to-text translation. This has led to astonishing improvements in this area. In contrast, the activities in the area of speech-to-speech translation is still limited, although it is essential to overcome the language barrier. We believe that one of the limiting factors is the availability of appropriate training data. We address this issue by creating LibriS2S, to our knowledge the first publicly available speech-to-speech training corpus between German and English. For this corpus, we used independently created audio for German and English leading to an unbiased pronunciation of the text in both languages. This allows the creation of a new text-to-speech and speech-to-speech translation model that directly learns to generate the speech signal based on the pronunciation of the source language. Using this created corpus, we propose Text-to-Speech models based on the example of the recently proposed FastSpeech 2 model that integrates source language information. We do this by adapting the model to take information such as the pitch, energy or transcript from the source speech as additional input.
Anthology ID:
2022.lrec-1.98
Volume:
Proceedings of the Thirteenth Language Resources and Evaluation Conference
Month:
June
Year:
2022
Address:
Marseille, France
Editors:
Nicoletta Calzolari, Frédéric Béchet, Philippe Blache, Khalid Choukri, Christopher Cieri, Thierry Declerck, Sara Goggi, Hitoshi Isahara, Bente Maegaard, Joseph Mariani, Hélène Mazo, Jan Odijk, Stelios Piperidis
Venue:
LREC
SIG:
Publisher:
European Language Resources Association
Note:
Pages:
928–935
Language:
URL:
https://aclanthology.org/2022.lrec-1.98
DOI:
Bibkey:
Cite (ACL):
Pedro Jeuris and Jan Niehues. 2022. LibriS2S: A German-English Speech-to-Speech Translation Corpus. In Proceedings of the Thirteenth Language Resources and Evaluation Conference, pages 928–935, Marseille, France. European Language Resources Association.
Cite (Informal):
LibriS2S: A German-English Speech-to-Speech Translation Corpus (Jeuris & Niehues, LREC 2022)
Copy Citation:
PDF:
https://aclanthology.org/2022.lrec-1.98.pdf
Code
 pedrodke/libris2s
Data
LibriS2SLibriVoxDeEn