Addressing Asymmetry in Multilingual Neural Machine Translation with Fuzzy Task Clustering

Qian Wang, Jiajun Zhang


Abstract
Multilingual neural machine translation (NMT) enables positive knowledge transfer among multiple translation tasks with a shared underlying model, but a unified multilingual model usually suffers from capacity bottleneck when tens or hundreds of languages are involved. A possible solution is to cluster languages and train individual model for each cluster. However, the existing clustering methods based on language similarity cannot handle the asymmetric problem in multilingual NMT, i.e., one translation task A can benefit from another translation task B but task B will be harmed by task A. To address this problem, we propose a fuzzy task clustering method for multilingual NMT. Specifically, we employ task affinity, defined as the loss change of one translation task caused by the training of another, as the clustering criterion. Next, we cluster the translation tasks based on the task affinity, such that tasks from the same cluster can benefit each other. For each cluster, we further find out a set of auxiliary translation tasks that benefit the tasks in this cluster. In this way, the model for each cluster is trained not only on the tasks in the cluster but also on the auxiliary tasks. We conduct extensive experiments for one-to-many, manyto-one, and many-to-many translation scenarios to verify the effectiveness of our method.
Anthology ID:
2022.coling-1.455
Volume:
Proceedings of the 29th International Conference on Computational Linguistics
Month:
October
Year:
2022
Address:
Gyeongju, Republic of Korea
Venue:
COLING
SIG:
Publisher:
International Committee on Computational Linguistics
Note:
Pages:
5129–5141
Language:
URL:
https://aclanthology.org/2022.coling-1.455
DOI:
Bibkey:
Cite (ACL):
Qian Wang and Jiajun Zhang. 2022. Addressing Asymmetry in Multilingual Neural Machine Translation with Fuzzy Task Clustering. In Proceedings of the 29th International Conference on Computational Linguistics, pages 5129–5141, Gyeongju, Republic of Korea. International Committee on Computational Linguistics.
Cite (Informal):
Addressing Asymmetry in Multilingual Neural Machine Translation with Fuzzy Task Clustering (Wang & Zhang, COLING 2022)
Copy Citation:
PDF:
https://aclanthology.org/2022.coling-1.455.pdf