Zero-Shot Translation Quality Estimation with Explicit Cross-Lingual Patterns

Lei Zhou, Liang Ding, Koichi Takeda


Abstract
This paper describes our submission of the WMT 2020 Shared Task on Sentence Level Direct Assessment, Quality Estimation (QE). In this study, we empirically reveal the mismatching issue when directly adopting BERTScore (Zhang et al., 2020) to QE. Specifically, there exist lots of mismatching errors between source sentence and translated candidate sentence with token pairwise similarity. In response to this issue, we propose to expose explicit cross lingual patterns, e.g. word alignments and generation score, to our proposed zero-shot models. Experiments show that our proposed QE model with explicit cross-lingual patterns could alleviate the mismatching issue, thereby improving the performance. Encouragingly, our zero-shot QE method could achieve comparable performance with supervised QE method, and even outperforms the supervised counterpart on 2 out of 6 directions. We expect our work could shed light on the zero-shot QE model improvement.
Anthology ID:
2020.wmt-1.125
Volume:
Proceedings of the Fifth Conference on Machine Translation
Month:
November
Year:
2020
Address:
Online
Editors:
Loïc Barrault, Ondřej Bojar, Fethi Bougares, Rajen Chatterjee, Marta R. Costa-jussà, Christian Federmann, Mark Fishel, Alexander Fraser, Yvette Graham, Paco Guzman, Barry Haddow, Matthias Huck, Antonio Jimeno Yepes, Philipp Koehn, André Martins, Makoto Morishita, Christof Monz, Masaaki Nagata, Toshiaki Nakazawa, Matteo Negri
Venue:
WMT
SIG:
SIGMT
Publisher:
Association for Computational Linguistics
Note:
Pages:
1068–1074
Language:
URL:
https://aclanthology.org/2020.wmt-1.125
DOI:
Bibkey:
Cite (ACL):
Lei Zhou, Liang Ding, and Koichi Takeda. 2020. Zero-Shot Translation Quality Estimation with Explicit Cross-Lingual Patterns. In Proceedings of the Fifth Conference on Machine Translation, pages 1068–1074, Online. Association for Computational Linguistics.
Cite (Informal):
Zero-Shot Translation Quality Estimation with Explicit Cross-Lingual Patterns (Zhou et al., WMT 2020)
Copy Citation:
PDF:
https://aclanthology.org/2020.wmt-1.125.pdf
Video:
 https://slideslive.com/38939669