Perceived vs. measured performance in the post-editing of suggestions from machine translation and translation memories

Carlos S.C. Teixeira


Abstract
This paper investigates the behaviour of ten professional translators when performing translation tasks with and without translation suggestions, and with and without translation metadata. The measured performances are then compared with the translators’ perceptions of their performances. The variables that are taken into consideration are time, edits and errors. Keystroke logging and screen recording are used to measure time and edits, an error score system is used to identify errors and post-performance interviews are used to assess participants’ perceptions. The study looks at the correlations between the translators’ perceptions and their actual performances, and tries to understand the reasons behind any discrepancies. Translators are found to prefer an environment with translation suggestions and translation metadata to an environment without metadata. This preference, however, does not always correlate with an improved performance. Task familiarity seems to be the most prominent factor responsible for the positive perceptions, rather than any intrinsic characteristics in the tasks. A certain prejudice against MT is also present in some of the comments.
Anthology ID:
2014.amta-wptp.4
Volume:
Proceedings of the 11th Conference of the Association for Machine Translation in the Americas
Month:
October 22-26
Year:
2014
Address:
Vancouver, Canada
Editors:
Sharon O'Brien, Michel Simard, Lucia Specia
Venue:
AMTA
SIG:
Publisher:
Association for Machine Translation in the Americas
Note:
Pages:
45–59
Language:
URL:
https://aclanthology.org/2014.amta-wptp.4
DOI:
Bibkey:
Cite (ACL):
Carlos S.C. Teixeira. 2014. Perceived vs. measured performance in the post-editing of suggestions from machine translation and translation memories. In Proceedings of the 11th Conference of the Association for Machine Translation in the Americas, pages 45–59, Vancouver, Canada. Association for Machine Translation in the Americas.
Cite (Informal):
Perceived vs. measured performance in the post-editing of suggestions from machine translation and translation memories (Teixeira, AMTA 2014)
Copy Citation:
PDF:
https://aclanthology.org/2014.amta-wptp.4.pdf