Estimating Machine Translation Post-Editing Effort with HTER

Lucia Specia, Atefeh Farzindar


Abstract
Although Machine Translation (MT) has been attracting more and more attention from the translation industry, the quality of current MT systems still requires humans to post-edit translations to ensure their quality. The time necessary to post-edit bad quality translations can be the same or even longer than that of translating without an MT system. It is well known, however, that the quality of an MT system is generally not homogeneous across all translated segments. In order to make MT more useful to the translation industry, it is therefore crucial to have a mechanism to judge MT quality at the segment level to prevent bad quality translations from being post-edited within the translation workflow. We describe an approach to estimate translation post-editing effort at sentence level in terms of Human-targeted Translation Edit Rate (HTER) based on a number of features reflecting the difficulty of translating the source sentence and discrepancies between the source and translation sentences. HTER is a simple metric and obtaining HTER annotated data can be made part of the translation workflow. We show that this approach is more reliable at filtering out bad translations than other simple criteria commonly used in the translation industry, such as sentence length.
Anthology ID:
2010.jec-1.5
Volume:
Proceedings of the Second Joint EM+/CNGL Workshop: Bringing MT to the User: Research on Integrating MT in the Translation Industry
Month:
November 4
Year:
2010
Address:
Denver, Colorado, USA
Venues:
AMTA | JEC
SIG:
Publisher:
Association for Machine Translation in the Americas
Note:
Pages:
33–43
Language:
URL:
https://aclanthology.org/2010.jec-1.5
DOI:
Bibkey:
Copy Citation:
PDF:
https://aclanthology.org/2010.jec-1.5.pdf