Translation Quality Estimation by Jointly Learning to Score and Rank

Jingyi Zhang, Josef van Genabith


Abstract
The translation quality estimation (QE) task, particularly the QE as a Metric task, aims to evaluate the general quality of a translation based on the translation and the source sentence without using reference translations. Supervised learning of this QE task requires human evaluation of translation quality as training data. Human evaluation of translation quality can be performed in different ways, including assigning an absolute score to a translation or ranking different translations. In order to make use of different types of human evaluation data for supervised learning, we present a multi-task learning QE model that jointly learns two tasks: score a translation and rank two translations. Our QE model exploits cross-lingual sentence embeddings from pre-trained multilingual language models. We obtain new state-of-the-art results on the WMT 2019 QE as a Metric task and outperform sentBLEU on the WMT 2019 Metrics task.
Anthology ID:
2020.emnlp-main.205
Volume:
Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)
Month:
November
Year:
2020
Address:
Online
Editors:
Bonnie Webber, Trevor Cohn, Yulan He, Yang Liu
Venue:
EMNLP
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
2592–2598
Language:
URL:
https://aclanthology.org/2020.emnlp-main.205
DOI:
10.18653/v1/2020.emnlp-main.205
Bibkey:
Cite (ACL):
Jingyi Zhang and Josef van Genabith. 2020. Translation Quality Estimation by Jointly Learning to Score and Rank. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 2592–2598, Online. Association for Computational Linguistics.
Cite (Informal):
Translation Quality Estimation by Jointly Learning to Score and Rank (Zhang & van Genabith, EMNLP 2020)
Copy Citation:
PDF:
https://aclanthology.org/2020.emnlp-main.205.pdf
Video:
 https://slideslive.com/38939037