Identifying the Correlation Between Language Distance and Cross-Lingual Transfer in a Multilingual Representation Space

Fred Philippy, Siwen Guo, Shohreh Haddadan


Abstract
Prior research has investigated the impact of various linguistic features on cross-lingual transfer performance. In this study, we investigate the manner in which this effect can be mapped onto the representation space. While past studies have focused on the impact on cross-lingual alignment in multilingual language models during fine-tuning, this study examines the absolute evolution of the respective language representation spaces produced by MLLMs. We place a specific emphasis on the role of linguistic characteristics and investigate their inter-correlation with the impact on representation spaces and cross-lingual transfer performance. Additionally, this paper provides preliminary evidence of how these findings can be leveraged to enhance transfer to linguistically distant languages.
Anthology ID:
2023.sigtyp-1.3
Volume:
Proceedings of the 5th Workshop on Research in Computational Linguistic Typology and Multilingual NLP
Month:
May
Year:
2023
Address:
Dubrovnik, Croatia
Venue:
SIGTYP
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
22–29
Language:
URL:
https://aclanthology.org/2023.sigtyp-1.3
DOI:
10.18653/v1/2023.sigtyp-1.3
Bibkey:
Cite (ACL):
Fred Philippy, Siwen Guo, and Shohreh Haddadan. 2023. Identifying the Correlation Between Language Distance and Cross-Lingual Transfer in a Multilingual Representation Space. In Proceedings of the 5th Workshop on Research in Computational Linguistic Typology and Multilingual NLP, pages 22–29, Dubrovnik, Croatia. Association for Computational Linguistics.
Cite (Informal):
Identifying the Correlation Between Language Distance and Cross-Lingual Transfer in a Multilingual Representation Space (Philippy et al., SIGTYP 2023)
Copy Citation:
PDF:
https://aclanthology.org/2023.sigtyp-1.3.pdf
Video:
 https://aclanthology.org/2023.sigtyp-1.3.mp4