Findings of the WMT 2020 Shared Task on Chat Translation

M. Amin Farajian, António V. Lopes, André F. T. Martins, Sameen Maruf, Gholamreza Haffari


Abstract
We report the results of the first edition of the WMT shared task on chat translation. The task consisted of translating bilingual conversational text, in particular customer support chats for the English-German language pair (English agent, German customer). This task varies from the other translation shared tasks, i.e. news and biomedical, mainly due to the fact that the conversations are bilingual, less planned, more informal, and often ungrammatical. Furthermore, such conversations are usually characterized by shorter and simpler sentences and contain more pronouns. We received 14 submissions from 6 participating teams, all of them covering both directions, i.e. En->De for agent utterances and De->En for customer messages. We used automatic metrics (BLEU and TER) for evaluating the translations of both agent and customer messages and human document-level direct assessments (DDA) to evaluate the agent translations.
Anthology ID:
2020.wmt-1.3
Volume:
Proceedings of the Fifth Conference on Machine Translation
Month:
November
Year:
2020
Address:
Online
Venue:
WMT
SIG:
SIGMT
Publisher:
Association for Computational Linguistics
Note:
Pages:
65–75
Language:
URL:
https://aclanthology.org/2020.wmt-1.3
DOI:
Bibkey:
Cite (ACL):
M. Amin Farajian, António V. Lopes, André F. T. Martins, Sameen Maruf, and Gholamreza Haffari. 2020. Findings of the WMT 2020 Shared Task on Chat Translation. In Proceedings of the Fifth Conference on Machine Translation, pages 65–75, Online. Association for Computational Linguistics.
Cite (Informal):
Findings of the WMT 2020 Shared Task on Chat Translation (Farajian et al., WMT 2020)
Copy Citation:
PDF:
https://aclanthology.org/2020.wmt-1.3.pdf
Video:
 https://slideslive.com/38939674
Code
 Unbabel/BConTrasT
Data
Taskmaster-1