Task-based evaluation methods for machine translation, in practice and theory

Judith L. Klavans


Abstract
A panel of industry and government experts will discuss ways in which they have applied task-based evaluation for Machine Translation and other language technologies in their organizations and share ideas for new methods that could be tried in the future. As part of the discussion, the panelists will address some of the following points: What task-based evaluation means within their organization, i.e., how task-based evaluation is defined; How task-based evaluation impacts the use of MT technologies in their work environment; Whether task-based evaluation correlates with MT developers' automated metrics and if not, how do we arrive at automated metrics that do correlate with the more expensive task-based evaluation; What "lessons-learned" resulted from the course of performing task-based evaluation; How task-based evaluations can be generalized to multiple workflow environments.
Anthology ID:
2010.amta-government.9
Volume:
Proceedings of the 9th Conference of the Association for Machine Translation in the Americas: Government MT User Program
Month:
October 31-November 4
Year:
2010
Address:
Denver, Colorado, USA
Venue:
AMTA
SIG:
Publisher:
Association for Machine Translation in the Americas
Note:
Pages:
Language:
URL:
https://aclanthology.org/2010.amta-government.9
DOI:
Bibkey:
Cite (ACL):
Judith L. Klavans. 2010. Task-based evaluation methods for machine translation, in practice and theory. In Proceedings of the 9th Conference of the Association for Machine Translation in the Americas: Government MT User Program, Denver, Colorado, USA. Association for Machine Translation in the Americas.
Cite (Informal):
Task-based evaluation methods for machine translation, in practice and theory (Klavans, AMTA 2010)
Copy Citation: