On Context Span Needed for Machine Translation Evaluation

Sheila Castilho, Maja Popović, Andy Way


Abstract
Despite increasing efforts to improve evaluation of machine translation (MT) by going beyond the sentence level to the document level, the definition of what exactly constitutes a “document level” is still not clear. This work deals with the context span necessary for a more reliable MT evaluation. We report results from a series of surveys involving three domains and 18 target languages designed to identify the necessary context span as well as issues related to it. Our findings indicate that, despite the fact that some issues and spans are strongly dependent on domain and on the target language, a number of common patterns can be observed so that general guidelines for context-aware MT evaluation can be drawn.
Anthology ID:
2020.lrec-1.461
Volume:
Proceedings of the Twelfth Language Resources and Evaluation Conference
Month:
May
Year:
2020
Address:
Marseille, France
Editors:
Nicoletta Calzolari, Frédéric Béchet, Philippe Blache, Khalid Choukri, Christopher Cieri, Thierry Declerck, Sara Goggi, Hitoshi Isahara, Bente Maegaard, Joseph Mariani, Hélène Mazo, Asuncion Moreno, Jan Odijk, Stelios Piperidis
Venue:
LREC
SIG:
Publisher:
European Language Resources Association
Note:
Pages:
3735–3742
Language:
English
URL:
https://aclanthology.org/2020.lrec-1.461
DOI:
Bibkey:
Cite (ACL):
Sheila Castilho, Maja Popović, and Andy Way. 2020. On Context Span Needed for Machine Translation Evaluation. In Proceedings of the Twelfth Language Resources and Evaluation Conference, pages 3735–3742, Marseille, France. European Language Resources Association.
Cite (Informal):
On Context Span Needed for Machine Translation Evaluation (Castilho et al., LREC 2020)
Copy Citation:
PDF:
https://aclanthology.org/2020.lrec-1.461.pdf