Two Phase Evaluation for Selecting Machine Translation Services

Chunqi Shi, Donghui Lin, Masahiko Shimada, Toru Ishida


Abstract
An increased number of machine translation services are now available. Unfortunately, none of them can provide adequate translation quality for all input sources. This forces the user to select from among the services according to his needs. However, it is tedious and time consuming to perform this manual selection. Our solution, proposed here, is an automatic mechanism that can select the most appropriate machine translation service. Although evaluation methods are available, such as BLEU, NIST, WER, etc., their evaluation results are not unanimous regardless of the translation sources. We proposed a two-phase architecture for selecting translation services. The first phase uses a data-driven classification to allow the most appropriate evaluation method to be selected according to each translation source. The second phase selects the most appropriate machine translation result by the selected evaluation method. We describe the architecture, detail the algorithm, and construct a prototype. Tests show that the proposal yields better translation quality than employing just one machine translation service.
Anthology ID:
L12-1472
Volume:
Proceedings of the Eighth International Conference on Language Resources and Evaluation (LREC'12)
Month:
May
Year:
2012
Address:
Istanbul, Turkey
Editors:
Nicoletta Calzolari, Khalid Choukri, Thierry Declerck, Mehmet Uğur Doğan, Bente Maegaard, Joseph Mariani, Asuncion Moreno, Jan Odijk, Stelios Piperidis
Venue:
LREC
SIG:
Publisher:
European Language Resources Association (ELRA)
Note:
Pages:
1771–1778
Language:
URL:
http://www.lrec-conf.org/proceedings/lrec2012/pdf/802_Paper.pdf
DOI:
Bibkey:
Cite (ACL):
Chunqi Shi, Donghui Lin, Masahiko Shimada, and Toru Ishida. 2012. Two Phase Evaluation for Selecting Machine Translation Services. In Proceedings of the Eighth International Conference on Language Resources and Evaluation (LREC'12), pages 1771–1778, Istanbul, Turkey. European Language Resources Association (ELRA).
Cite (Informal):
Two Phase Evaluation for Selecting Machine Translation Services (Shi et al., LREC 2012)
Copy Citation:
PDF:
http://www.lrec-conf.org/proceedings/lrec2012/pdf/802_Paper.pdf