Modeling User Preferences with Automatic Metrics: Creating a High-Quality Preference Dataset for Machine Translation

Sweta Agrawal, José G. C. De Souza, Ricardo Rei, António Farinhas, Gonçalo Faria, Patrick Fernandes, Nuno M Guerreiro, Andre Martins


Abstract
Alignment with human preferences is an important step in developing accurate and safe large language models. This is no exception in machine translation (MT), where better handling of language nuances and context-specific variations leads to improved quality. However, preference data based on human feedback can be very expensive to obtain and curate at a large scale. Automatic metrics, on the other hand, can induce preferences, but they might not match human expectations perfectly. In this paper, we propose an approach that leverages the best of both worlds. We first collect sentence-level quality assessments from professional linguists on translations generated by multiple high-quality MT systems and evaluate the ability of current automatic metrics to recover these preferences. We then use this analysis to curate a new dataset, MT-Pref (metric induced translation preference) dataset, which comprises 18k instances covering 18 language directions, using texts sourced from multiple domains post-2022. We show that aligning TOWER models on MT-Pref significantly improves translation quality on WMT23 and FLORES benchmarks.
Anthology ID:
2024.emnlp-main.803
Volume:
Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing
Month:
November
Year:
2024
Address:
Miami, Florida, USA
Editors:
Yaser Al-Onaizan, Mohit Bansal, Yun-Nung Chen
Venue:
EMNLP
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
14503–14519
Language:
URL:
https://aclanthology.org/2024.emnlp-main.803
DOI:
10.18653/v1/2024.emnlp-main.803
Bibkey:
Cite (ACL):
Sweta Agrawal, José G. C. De Souza, Ricardo Rei, António Farinhas, Gonçalo Faria, Patrick Fernandes, Nuno M Guerreiro, and Andre Martins. 2024. Modeling User Preferences with Automatic Metrics: Creating a High-Quality Preference Dataset for Machine Translation. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 14503–14519, Miami, Florida, USA. Association for Computational Linguistics.
Cite (Informal):
Modeling User Preferences with Automatic Metrics: Creating a High-Quality Preference Dataset for Machine Translation (Agrawal et al., EMNLP 2024)
Copy Citation:
PDF:
https://aclanthology.org/2024.emnlp-main.803.pdf