Automatically predicting MT systems rankings compatible with fluency, adequacy and informativeness scores

Martin Rajman, Tony Hartley


Abstract
The main goal of the work presented in this paper is to find an inexpensive and automatable way of predicting rankings of MT systems compatible with human evaluations of these systems expressed in the form of Fluency, Adequacy or Informativeness scores. Our approach is to establish whether there is a correlation between rankings derived from such scores and the ones that can be built on the basis of automatically computable attributes of syntactic or semantic nature. We present promising results obtained on the DARPA94 MT evaluation corpus.
Anthology ID:
2001.mtsummit-eval.6
Volume:
Workshop on MT Evaluation
Month:
September 18-22
Year:
2001
Address:
Santiago de Compostela, Spain
Editors:
Eduard Hovy, Margaret King, Sandra Manzi, Florence Reeder
Venue:
MTSummit
SIG:
Publisher:
Note:
Pages:
Language:
URL:
https://aclanthology.org/2001.mtsummit-eval.6
DOI:
Bibkey:
Cite (ACL):
Martin Rajman and Tony Hartley. 2001. Automatically predicting MT systems rankings compatible with fluency, adequacy and informativeness scores. In Workshop on MT Evaluation, Santiago de Compostela, Spain.
Cite (Informal):
Automatically predicting MT systems rankings compatible with fluency, adequacy and informativeness scores (Rajman & Hartley, MTSummit 2001)
Copy Citation:
PDF:
https://aclanthology.org/2001.mtsummit-eval.6.pdf