A User-Based Usability Assessment of Raw Machine Translated Technical Instructions

Stephen Doherty, Sharon O’Brien


Abstract
This paper reports on a project whose aims are to investigate the usability of raw machine translated technical support documentation for a commercial online file storage service. Following the ISO/TR 16982 definition of usability - goal completion, satisfaction, effectiveness, and efficiency - comparisons are drawn for all measures between the original user documentation written in English for a well-known online file storage service and raw machine translated output in four target languages: Spanish, French, German and Japanese. Using native speakers for each language, we found significant differences between the source and MT output for three out of the four measures: goal completion, efficiency and user satisfaction. This leads to a tentative conclusion that there is a difference in usability between well-formed content and raw machine translated content, and we suggest avenues for further work.
Anthology ID:
2012.amta-commercial.4
Volume:
Proceedings of the 10th Conference of the Association for Machine Translation in the Americas: Commercial MT User Program
Month:
October 28-November 1
Year:
2012
Address:
San Diego, California, USA
Venue:
AMTA
SIG:
Publisher:
Association for Machine Translation in the Americas
Note:
Pages:
Language:
URL:
https://aclanthology.org/2012.amta-commercial.4
DOI:
Bibkey:
Cite (ACL):
Stephen Doherty and Sharon O’Brien. 2012. A User-Based Usability Assessment of Raw Machine Translated Technical Instructions. In Proceedings of the 10th Conference of the Association for Machine Translation in the Americas: Commercial MT User Program, San Diego, California, USA. Association for Machine Translation in the Americas.
Cite (Informal):
A User-Based Usability Assessment of Raw Machine Translated Technical Instructions (Doherty & O’Brien, AMTA 2012)
Copy Citation:
PDF:
https://aclanthology.org/2012.amta-commercial.4.pdf