Sentence Level Human Translation Quality Estimation with Attention-based Neural Networks

Yu Yuan, Serge Sharoff


Abstract
This paper explores the use of Deep Learning methods for automatic estimation of quality of human translations. Automatic estimation can provide useful feedback for translation teaching, examination and quality control. Conventional methods for solving this task rely on manually engineered features and external knowledge. This paper presents an end-to-end neural model without feature engineering, incorporating a cross attention mechanism to detect which parts in sentence pairs are most relevant for assessing quality. Another contribution concerns oprediction of fine-grained scores for measuring different aspects of translation quality, such as terminological accuracy or idiomatic writing. Empirical results on a large human annotated dataset show that the neural model outperforms feature-based methods significantly. The dataset and the tools are available.
Anthology ID:
2020.lrec-1.229
Volume:
Proceedings of the Twelfth Language Resources and Evaluation Conference
Month:
May
Year:
2020
Address:
Marseille, France
Editors:
Nicoletta Calzolari, Frédéric Béchet, Philippe Blache, Khalid Choukri, Christopher Cieri, Thierry Declerck, Sara Goggi, Hitoshi Isahara, Bente Maegaard, Joseph Mariani, Hélène Mazo, Asuncion Moreno, Jan Odijk, Stelios Piperidis
Venue:
LREC
SIG:
Publisher:
European Language Resources Association
Note:
Pages:
1858–1865
Language:
English
URL:
https://aclanthology.org/2020.lrec-1.229
DOI:
Bibkey:
Cite (ACL):
Yu Yuan and Serge Sharoff. 2020. Sentence Level Human Translation Quality Estimation with Attention-based Neural Networks. In Proceedings of the Twelfth Language Resources and Evaluation Conference, pages 1858–1865, Marseille, France. European Language Resources Association.
Cite (Informal):
Sentence Level Human Translation Quality Estimation with Attention-based Neural Networks (Yuan & Sharoff, LREC 2020)
Copy Citation:
PDF:
https://aclanthology.org/2020.lrec-1.229.pdf