DELA Corpus - A Document-Level Corpus Annotated with Context-Related Issues

Sheila Castilho, João Lucas Cavalheiro Camargo, Miguel Menezes, Andy Way


Abstract
Recently, the Machine Translation (MT) community has become more interested in document-level evaluation especially in light of reactions to claims of “human parity”, since examining the quality at the level of the document rather than at the sentence level allows for the assessment of suprasentential context, providing a more reliable evaluation. This paper presents a document-level corpus annotated in English with context-aware issues that arise when translating from English into Brazilian Portuguese, namely ellipsis, gender, lexical ambiguity, number, reference, and terminology, with six different domains. The corpus can be used as a challenge test set for evaluation and as a training/testing corpus for MT as well as for deep linguistic analysis of context issues. To the best of our knowledge, this is the first corpus of its kind.
Anthology ID:
2021.wmt-1.63
Volume:
Proceedings of the Sixth Conference on Machine Translation
Month:
November
Year:
2021
Address:
Online
Venues:
EMNLP | WMT
SIG:
SIGMT
Publisher:
Association for Computational Linguistics
Note:
Pages:
566–577
Language:
URL:
https://aclanthology.org/2021.wmt-1.63
DOI:
Bibkey:
Cite (ACL):
Sheila Castilho, João Lucas Cavalheiro Camargo, Miguel Menezes, and Andy Way. 2021. DELA Corpus - A Document-Level Corpus Annotated with Context-Related Issues. In Proceedings of the Sixth Conference on Machine Translation, pages 566–577, Online. Association for Computational Linguistics.
Cite (Informal):
DELA Corpus - A Document-Level Corpus Annotated with Context-Related Issues (Castilho et al., WMT 2021)
Copy Citation:
PDF:
https://aclanthology.org/2021.wmt-1.63.pdf