An Exploratory Analysis of Multilingual Word-Level Quality Estimation with Cross-Lingual Transformers

Tharindu Ranasinghe, Constantin Orasan, Ruslan Mitkov


Abstract
Most studies on word-level Quality Estimation (QE) of machine translation focus on language-specific models. The obvious disadvantages of these approaches are the need for labelled data for each language pair and the high cost required to maintain several language-specific models. To overcome these problems, we explore different approaches to multilingual, word-level QE. We show that multilingual QE models perform on par with the current language-specific models. In the cases of zero-shot and few-shot QE, we demonstrate that it is possible to accurately predict word-level quality for any given new language pair from models trained on other language pairs. Our findings suggest that the word-level QE models based on powerful pre-trained transformers that we propose in this paper generalise well across languages, making them more useful in real-world scenarios.
Anthology ID:
2021.acl-short.55
Volume:
Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 2: Short Papers)
Month:
August
Year:
2021
Address:
Online
Editors:
Chengqing Zong, Fei Xia, Wenjie Li, Roberto Navigli
Venues:
ACL | IJCNLP
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
434–440
Language:
URL:
https://aclanthology.org/2021.acl-short.55
DOI:
10.18653/v1/2021.acl-short.55
Bibkey:
Cite (ACL):
Tharindu Ranasinghe, Constantin Orasan, and Ruslan Mitkov. 2021. An Exploratory Analysis of Multilingual Word-Level Quality Estimation with Cross-Lingual Transformers. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 2: Short Papers), pages 434–440, Online. Association for Computational Linguistics.
Cite (Informal):
An Exploratory Analysis of Multilingual Word-Level Quality Estimation with Cross-Lingual Transformers (Ranasinghe et al., ACL-IJCNLP 2021)
Copy Citation:
PDF:
https://aclanthology.org/2021.acl-short.55.pdf
Optional supplementary material:
 2021.acl-short.55.OptionalSupplementaryMaterial.pdf
Video:
 https://aclanthology.org/2021.acl-short.55.mp4
Code
 tharindudr/transQuest