A Systematic Analysis of Subwords and Cross-Lingual Transfer in Multilingual Translation

Francois Meyer, Jan Buys


Abstract
Multilingual modelling can improve machine translation for low-resource languages, partly through shared subword representations. This paper studies the role of subword segmentation in cross-lingual transfer. We systematically compare the efficacy of several subword methods in promoting synergy and preventing interference across different linguistic typologies. Our findings show that subword regularisation boosts synergy in multilingual modelling, whereas BPE more effectively facilitates transfer during cross-lingual fine-tuning. Notably, our results suggest that differences in orthographic word boundary conventions (the morphological granularity of written words) may impede cross-lingual transfer more significantly than linguistic unrelatedness. Our study confirms that decisions around subword modelling can be key to optimising the benefits of multilingual modelling.
Anthology ID:
2024.findings-naacl.141
Volume:
Findings of the Association for Computational Linguistics: NAACL 2024
Month:
June
Year:
2024
Address:
Mexico City, Mexico
Editors:
Kevin Duh, Helena Gomez, Steven Bethard
Venue:
Findings
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
2194–2200
Language:
URL:
https://aclanthology.org/2024.findings-naacl.141
DOI:
Bibkey:
Cite (ACL):
Francois Meyer and Jan Buys. 2024. A Systematic Analysis of Subwords and Cross-Lingual Transfer in Multilingual Translation. In Findings of the Association for Computational Linguistics: NAACL 2024, pages 2194–2200, Mexico City, Mexico. Association for Computational Linguistics.
Cite (Informal):
A Systematic Analysis of Subwords and Cross-Lingual Transfer in Multilingual Translation (Meyer & Buys, Findings 2024)
Copy Citation:
PDF:
https://aclanthology.org/2024.findings-naacl.141.pdf
Copyright:
 2024.findings-naacl.141.copyright.pdf