Document-level Neural MT: A Systematic Comparison

António Lopes, M. Amin Farajian, Rachel Bawden, Michael Zhang, André F. T. Martins


Abstract
In this paper we provide a systematic comparison of existing and new document-level neural machine translation solutions. As part of this comparison, we introduce and evaluate a document-level variant of the recently proposed Star Transformer architecture. In addition to using the traditional metric BLEU, we report the accuracy of the models in handling anaphoric pronoun translation as well as coherence and cohesion using contrastive test sets. Finally, we report the results of human evaluation in terms of Multidimensional Quality Metrics (MQM) and analyse the correlation of the results obtained by the automatic metrics with human judgments.
Anthology ID:
2020.eamt-1.24
Volume:
Proceedings of the 22nd Annual Conference of the European Association for Machine Translation
Month:
November
Year:
2020
Address:
Lisboa, Portugal
Venue:
EAMT
SIG:
Publisher:
European Association for Machine Translation
Note:
Pages:
225–234
Language:
URL:
https://aclanthology.org/2020.eamt-1.24
DOI:
Bibkey:
Copy Citation:
PDF:
https://aclanthology.org/2020.eamt-1.24.pdf