Granularity in MT evaluation

Florence Reeder, John White


Abstract
This paper looks at granularity issues in machine translation evaluation. We start with work by (White, 2001) who examined the correlation between intelligibility and fidelity at the document level. His work showed that intelligibility and fidelity do not correlate well at the document level. These dissimilarities lead to our investigation of evaluation granularity. In particular, we revisit the intelligibility and fidelity relationship at the corpus level. We expect these to support certain assumptions in both evaluations as well as indicate issues germane to future evaluations.
Anthology ID:
2003.mtsummit-eval.5
Volume:
Workshop on Systemizing MT Evaluation
Month:
September 23-27
Year:
2003
Address:
New Orleans, USA
Venue:
MTSummit
SIG:
Publisher:
Note:
Pages:
Language:
URL:
https://aclanthology.org/2003.mtsummit-eval.5
DOI:
Bibkey:
Copy Citation:
PDF:
https://aclanthology.org/2003.mtsummit-eval.5.pdf