MEE4 and XLsim : IIIT HYD’s Submissions’ for WMT23 Metrics Shared Task

Ananya Mukherjee, Manish Shrivastava


Abstract
This paper presents our contributions to the WMT2023 shared metrics task, consisting of two distinct evaluation approaches: a) Unsupervised Metric (MEE4) and b) Supervised Metric (XLSim). MEE4 represents an unsupervised, reference-based assessment metric that quantifies linguistic features, encompassing lexical, syntactic, semantic, morphological, and contextual similarities, leveraging embeddings. In contrast, XLsim is a supervised reference-based evaluation metric, employing a Siamese Architecture, which regresses on Direct Assessments (DA) from previous WMT News Translation shared tasks from 2017-2022. XLsim is trained using XLM-RoBERTa (base) on English-German reference and mt pairs with human scores.
Anthology ID:
2023.wmt-1.66
Volume:
Proceedings of the Eighth Conference on Machine Translation
Month:
December
Year:
2023
Address:
Singapore
Editors:
Philipp Koehn, Barry Haddow, Tom Kocmi, Christof Monz
Venue:
WMT
SIG:
SIGMT
Publisher:
Association for Computational Linguistics
Note:
Pages:
800–805
Language:
URL:
https://aclanthology.org/2023.wmt-1.66
DOI:
10.18653/v1/2023.wmt-1.66
Bibkey:
Cite (ACL):
Ananya Mukherjee and Manish Shrivastava. 2023. MEE4 and XLsim : IIIT HYD’s Submissions’ for WMT23 Metrics Shared Task. In Proceedings of the Eighth Conference on Machine Translation, pages 800–805, Singapore. Association for Computational Linguistics.
Cite (Informal):
MEE4 and XLsim : IIIT HYD’s Submissions’ for WMT23 Metrics Shared Task (Mukherjee & Shrivastava, WMT 2023)
Copy Citation:
PDF:
https://aclanthology.org/2023.wmt-1.66.pdf