Vocabulary-informed Language Encoding

Xi Ai, Bin Fang


Abstract
A Multilingual model relies on language encodings to identify input languages because the multilingual model has to distinguish between the input and output languages or among all the languages for cross-lingual tasks. Furthermore, we find that language encodings potentially refine multiple morphologies of different languages to form a better isomorphic space for multilinguality. To leverage this observation, we present a method to compute a vocabulary-informed language encoding as the language representation, for a required language, considering a local vocabulary covering an acceptable amount of the most frequent word embeddings in this language. In our experiments, our method can consistently improve the performance of multilingual models on unsupervised neural machine translation and cross-lingual embedding.
Anthology ID:
2022.coling-1.432
Volume:
Proceedings of the 29th International Conference on Computational Linguistics
Month:
October
Year:
2022
Address:
Gyeongju, Republic of Korea
Venue:
COLING
SIG:
Publisher:
International Committee on Computational Linguistics
Note:
Pages:
4883–4891
Language:
URL:
https://aclanthology.org/2022.coling-1.432
DOI:
Bibkey:
Cite (ACL):
Xi Ai and Bin Fang. 2022. Vocabulary-informed Language Encoding. In Proceedings of the 29th International Conference on Computational Linguistics, pages 4883–4891, Gyeongju, Republic of Korea. International Committee on Computational Linguistics.
Cite (Informal):
Vocabulary-informed Language Encoding (Ai & Fang, COLING 2022)
Copy Citation:
PDF:
https://aclanthology.org/2022.coling-1.432.pdf