ChrEnTranslate: Cherokee-English Machine Translation Demo with Quality Estimation and Corrective Feedback

Shiyue Zhang, Benjamin Frey, Mohit Bansal


Abstract
We introduce ChrEnTranslate, an online machine translation demonstration system for translation between English and an endangered language Cherokee. It supports both statistical and neural translation models as well as provides quality estimation to inform users of reliability, two user feedback interfaces for experts and common users respectively, example inputs to collect human translations for monolingual data, word alignment visualization, and relevant terms from the Cherokee English dictionary. The quantitative evaluation demonstrates that our backbone translation models achieve state-of-the-art translation performance and our quality estimation well correlates with both BLEU and human judgment. By analyzing 216 pieces of expert feedback, we find that NMT is preferable because it copies less than SMT, and, in general, current models can translate fragments of the source sentence but make major mistakes. When we add these 216 expert-corrected parallel texts into the training set and retrain models, equal or slightly better performance is observed, which demonstrates indicates the potential of human-in-the-loop learning.
Anthology ID:
2021.acl-demo.33
Volume:
Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing: System Demonstrations
Month:
August
Year:
2021
Address:
Online
Editors:
Heng Ji, Jong C. Park, Rui Xia
Venues:
ACL | IJCNLP
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
272–279
Language:
URL:
https://aclanthology.org/2021.acl-demo.33
DOI:
10.18653/v1/2021.acl-demo.33
Bibkey:
Cite (ACL):
Shiyue Zhang, Benjamin Frey, and Mohit Bansal. 2021. ChrEnTranslate: Cherokee-English Machine Translation Demo with Quality Estimation and Corrective Feedback. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing: System Demonstrations, pages 272–279, Online. Association for Computational Linguistics.
Cite (Informal):
ChrEnTranslate: Cherokee-English Machine Translation Demo with Quality Estimation and Corrective Feedback (Zhang et al., ACL-IJCNLP 2021)
Copy Citation:
PDF:
https://aclanthology.org/2021.acl-demo.33.pdf
Video:
 https://aclanthology.org/2021.acl-demo.33.mp4
Code
 ZhangShiyue/ChrEn +  additional community code