Backtranslation Feedback Improves User Confidence in MT, Not Quality

Vilém Zouhar, Michal Novák, Matúš Žilinec, Ondřej Bojar, Mateo Obregón, Robin L. Hill, Frédéric Blain, Marina Fomicheva, Lucia Specia, Lisa Yankovskaya


Abstract
Translating text into a language unknown to the text’s author, dubbed outbound translation, is a modern need for which the user experience has significant room for improvement, beyond the basic machine translation facility. We demonstrate this by showing three ways in which user confidence in the outbound translation, as well as its overall final quality, can be affected: backward translation, quality estimation (with alignment) and source paraphrasing. In this paper, we describe an experiment on outbound translation from English to Czech and Estonian. We examine the effects of each proposed feedback module and further focus on how the quality of machine translation systems influence these findings and the user perception of success. We show that backward translation feedback has a mixed effect on the whole process: it increases user confidence in the produced translation, but not the objective quality.
Anthology ID:
2021.naacl-main.14
Volume:
Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies
Month:
June
Year:
2021
Address:
Online
Venue:
NAACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
151–161
Language:
URL:
https://aclanthology.org/2021.naacl-main.14
DOI:
10.18653/v1/2021.naacl-main.14
Bibkey:
Copy Citation:
PDF:
https://aclanthology.org/2021.naacl-main.14.pdf
Optional supplementary code:
 2021.naacl-main.14.OptionalSupplementaryCode.zip
Code
 zouharvi/ptakopet