Findings of the WMT 2023 Shared Task on Automatic Post-Editing

Pushpak Bhattacharyya, Rajen Chatterjee, Markus Freitag, Diptesh Kanojia, Matteo Negri, Marco Turchi


Abstract
We present the results from the 9th round of the WMT shared task on MT Automatic Post-Editing, which consists of automatically correcting the output of a “black-box” machine translation system by learning from human corrections. Like last year, the task focused on English→Marathi, with data coming from multiple domains (healthcare, tourism, and general/news). Despite the consistent task framework, this year’s data proved to be extremely challenging. As a matter of fact, none of the official submissions from the participating teams succeeded in improving the quality of the already high-level initial translations (with baseline TER and BLEU scores of 26.6 and 70.66, respectively). Only one run, accepted as a “late” submission, achieved automatic evaluation scores that exceeded the baseline.
Anthology ID:
2023.wmt-1.55
Volume:
Proceedings of the Eighth Conference on Machine Translation
Month:
December
Year:
2023
Address:
Singapore
Editors:
Philipp Koehn, Barry Haddow, Tom Kocmi, Christof Monz
Venue:
WMT
SIG:
SIGMT
Publisher:
Association for Computational Linguistics
Note:
Pages:
672–681
Language:
URL:
https://aclanthology.org/2023.wmt-1.55
DOI:
10.18653/v1/2023.wmt-1.55
Bibkey:
Cite (ACL):
Pushpak Bhattacharyya, Rajen Chatterjee, Markus Freitag, Diptesh Kanojia, Matteo Negri, and Marco Turchi. 2023. Findings of the WMT 2023 Shared Task on Automatic Post-Editing. In Proceedings of the Eighth Conference on Machine Translation, pages 672–681, Singapore. Association for Computational Linguistics.
Cite (Informal):
Findings of the WMT 2023 Shared Task on Automatic Post-Editing (Bhattacharyya et al., WMT 2023)
Copy Citation:
PDF:
https://aclanthology.org/2023.wmt-1.55.pdf