An automatic evaluation method of translation quality using translation answer candidates queried from a parallel corpus

Keiji Yasuda, Fumiaki Sugaya, Toshiyuki Takezawa, Seiichi Yamamoto, Masuzo Yanagida


Abstract
An automatic translation quality evaluation method is proposed. In the proposed method, a parallel corpus is used to query translation answer candidates. The translation output is evaluated by measuring the similarity between the translation output and translation answer candidates with DP matching. This method evaluates a language translation subsystem of the Japanese-to-English ATR-MATRIX speech translation system developed at ATR Interpreting Telecommunications Research Laboratories. Discriminant analysis is then carried out to examine the evaluation performance of the proposed method. Experimental results show the effectiveness of the proposed method. The discriminant ratio is 83.5% for 2-class discrimination between absolutely correct and less appropriate translations classified subjectively. Also discussed are issues of the proposed method when it is applied to speech translation systems which inevitably make recognition errors.
Anthology ID:
2001.mtsummit-papers.67
Volume:
Proceedings of Machine Translation Summit VIII
Month:
September 18-22
Year:
2001
Address:
Santiago de Compostela, Spain
Editor:
Bente Maegaard
Venue:
MTSummit
SIG:
Publisher:
Note:
Pages:
Language:
URL:
https://aclanthology.org/2001.mtsummit-papers.67
DOI:
Bibkey:
Cite (ACL):
Keiji Yasuda, Fumiaki Sugaya, Toshiyuki Takezawa, Seiichi Yamamoto, and Masuzo Yanagida. 2001. An automatic evaluation method of translation quality using translation answer candidates queried from a parallel corpus. In Proceedings of Machine Translation Summit VIII, Santiago de Compostela, Spain.
Cite (Informal):
An automatic evaluation method of translation quality using translation answer candidates queried from a parallel corpus (Yasuda et al., MTSummit 2001)
Copy Citation:
PDF:
https://aclanthology.org/2001.mtsummit-papers.67.pdf