Bayesian Model-Agnostic Meta-Learning with Matrix-Valued Kernels for Quality Estimation

Abiola Obamuyide, Marina Fomicheva, Lucia Specia


Abstract
Most current quality estimation (QE) models for machine translation are trained and evaluated in a fully supervised setting requiring significant quantities of labelled training data. However, obtaining labelled data can be both expensive and time-consuming. In addition, the test data that a deployed QE model would be exposed to may differ from its training data in significant ways. In particular, training samples are often labelled by one or a small set of annotators, whose perceptions of translation quality and needs may differ substantially from those of end-users, who will employ predictions in practice. Thus, it is desirable to be able to adapt QE models efficiently to new user data with limited supervision data. To address these challenges, we propose a Bayesian meta-learning approach for adapting QE models to the needs and preferences of each user with limited supervision. To enhance performance, we further propose an extension to a state-of-the-art Bayesian meta-learning approach which utilizes a matrix-valued kernel for Bayesian meta-learning of quality estimation. Experiments on data with varying number of users and language characteristics demonstrates that the proposed Bayesian meta-learning approach delivers improved predictive performance in both limited and full supervision settings.
Anthology ID:
2021.repl4nlp-1.23
Volume:
Proceedings of the 6th Workshop on Representation Learning for NLP (RepL4NLP-2021)
Month:
August
Year:
2021
Address:
Online
Editors:
Anna Rogers, Iacer Calixto, Ivan Vulić, Naomi Saphra, Nora Kassner, Oana-Maria Camburu, Trapit Bansal, Vered Shwartz
Venue:
RepL4NLP
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
223–230
Language:
URL:
https://aclanthology.org/2021.repl4nlp-1.23
DOI:
10.18653/v1/2021.repl4nlp-1.23
Bibkey:
Cite (ACL):
Abiola Obamuyide, Marina Fomicheva, and Lucia Specia. 2021. Bayesian Model-Agnostic Meta-Learning with Matrix-Valued Kernels for Quality Estimation. In Proceedings of the 6th Workshop on Representation Learning for NLP (RepL4NLP-2021), pages 223–230, Online. Association for Computational Linguistics.
Cite (Informal):
Bayesian Model-Agnostic Meta-Learning with Matrix-Valued Kernels for Quality Estimation (Obamuyide et al., RepL4NLP 2021)
Copy Citation:
PDF:
https://aclanthology.org/2021.repl4nlp-1.23.pdf