Exploring the Importance of Source Text in Automatic Post-Editing for Context-Aware Machine Translation

Chaojun Wang, Christian Hardmeier, Rico Sennrich


Abstract
Accurate translation requires document-level information, which is ignored by sentence-level machine translation. Recent work has demonstrated that document-level consistency can be improved with automatic post-editing (APE) using only target-language (TL) information. We study an extended APE model that additionally integrates source context. A human evaluation of fluency and adequacy in English–Russian translation reveals that the model with access to source context significantly outperforms monolingual APE in terms of adequacy, an effect largely ignored by automatic evaluation metrics. Our results show that TL-only modelling increases fluency without improving adequacy, demonstrating the need for conditioning on source text for automatic post-editing. They also highlight blind spots in automatic methods for targeted evaluation and demonstrate the need for human assessment to evaluate document-level translation quality reliably.
Anthology ID:
2021.nodalida-main.34
Volume:
Proceedings of the 23rd Nordic Conference on Computational Linguistics (NoDaLiDa)
Month:
May 31--2 June
Year:
2021
Address:
Reykjavik, Iceland (Online)
Editors:
Simon Dobnik, Lilja Øvrelid
Venue:
NoDaLiDa
SIG:
Publisher:
Linköping University Electronic Press, Sweden
Note:
Pages:
326–335
Language:
URL:
https://aclanthology.org/2021.nodalida-main.34
DOI:
Bibkey:
Cite (ACL):
Chaojun Wang, Christian Hardmeier, and Rico Sennrich. 2021. Exploring the Importance of Source Text in Automatic Post-Editing for Context-Aware Machine Translation. In Proceedings of the 23rd Nordic Conference on Computational Linguistics (NoDaLiDa), pages 326–335, Reykjavik, Iceland (Online). Linköping University Electronic Press, Sweden.
Cite (Informal):
Exploring the Importance of Source Text in Automatic Post-Editing for Context-Aware Machine Translation (Wang et al., NoDaLiDa 2021)
Copy Citation:
PDF:
https://aclanthology.org/2021.nodalida-main.34.pdf
Code
 zippotju/context-aware-bilingual-repair-for-neural-machine-translation