Evaluating and Improving the Coreference Capabilities of Machine Translation Models

Asaf Yehudai, Arie Cattan, Omri Abend, Gabriel Stanovsky


Abstract
Machine translation (MT) requires a wide range of linguistic capabilities, which current end-to-end models are expected to learn implicitly by observing aligned sentences in bilingual corpora. In this work, we ask: How well MT models learn coreference resolution via implicit signal? To answer this question, we develop an evaluation methodology that derives coreference clusters from MT output and evaluates them without requiring annotations in the target language.Following, we evaluate several prominent open-source and commercial MT systems, translating from English to six target languages, and compare them to state-of-the-art coreference resolvers on three challenging benchmarks. Our results show that the monolingual resolvers greatly outperform MT models. Motivated by this result, we experiment with different methods for incorporating the output of coreference resolution models in MT, showing improvement over strong baselines.
Anthology ID:
2023.eacl-main.69
Volume:
Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics
Month:
May
Year:
2023
Address:
Dubrovnik, Croatia
Editors:
Andreas Vlachos, Isabelle Augenstein
Venue:
EACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
980–992
Language:
URL:
https://aclanthology.org/2023.eacl-main.69
DOI:
10.18653/v1/2023.eacl-main.69
Bibkey:
Cite (ACL):
Asaf Yehudai, Arie Cattan, Omri Abend, and Gabriel Stanovsky. 2023. Evaluating and Improving the Coreference Capabilities of Machine Translation Models. In Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics, pages 980–992, Dubrovnik, Croatia. Association for Computational Linguistics.
Cite (Informal):
Evaluating and Improving the Coreference Capabilities of Machine Translation Models (Yehudai et al., EACL 2023)
Copy Citation:
PDF:
https://aclanthology.org/2023.eacl-main.69.pdf
Video:
 https://aclanthology.org/2023.eacl-main.69.mp4