Report on the 11th IWSLT evaluation campaign

Mauro Cettolo, Jan Niehues, Sebastian Stüker, Luisa Bentivogli, Marcello Federico


Abstract
The paper overviews the 11th evaluation campaign organized by the IWSLT workshop. The 2014 evaluation offered multiple tracks on lecture transcription and translation based on the TED Talks corpus. In particular, this year IWSLT included three automatic speech recognition tracks, on English, German and Italian, five speech translation tracks, from English to French, English to German, German to English, English to Italian, and Italian to English, and five text translation track, also from English to French, English to German, German to English, English to Italian, and Italian to English. In addition to the official tracks, speech and text translation optional tracks were offered, globally involving 12 other languages: Arabic, Spanish, Portuguese (B), Hebrew, Chinese, Polish, Persian, Slovenian, Turkish, Dutch, Romanian, Russian. Overall, 21 teams participated in the evaluation, for a total of 76 primary runs submitted. Participants were also asked to submit runs on the 2013 test set (progress test set), in order to measure the progress of systems with respect to the previous year. All runs were evaluated with objective metrics, and submissions for two of the official text translation tracks were also evaluated with human post-editing.
Anthology ID:
2014.iwslt-evaluation.1
Volume:
Proceedings of the 11th International Workshop on Spoken Language Translation: Evaluation Campaign
Month:
December 4-5
Year:
2014
Address:
Lake Tahoe, California
Editors:
Marcello Federico, Sebastian Stüker, François Yvon
Venue:
IWSLT
SIG:
SIGSLT
Publisher:
Note:
Pages:
2–17
Language:
URL:
https://aclanthology.org/2014.iwslt-evaluation.1
DOI:
Bibkey:
Cite (ACL):
Mauro Cettolo, Jan Niehues, Sebastian Stüker, Luisa Bentivogli, and Marcello Federico. 2014. Report on the 11th IWSLT evaluation campaign. In Proceedings of the 11th International Workshop on Spoken Language Translation: Evaluation Campaign, pages 2–17, Lake Tahoe, California.
Cite (Informal):
Report on the 11th IWSLT evaluation campaign (Cettolo et al., IWSLT 2014)
Copy Citation:
PDF:
https://aclanthology.org/2014.iwslt-evaluation.1.pdf