BAD-X: Bilingual Adapters Improve Zero-Shot Cross-Lingual Transfer

Marinela Parović, Goran Glavaš, Ivan Vulić, Anna Korhonen


Abstract
Adapter modules enable modular and efficient zero-shot cross-lingual transfer, where current state-of-the-art adapter-based approaches learn specialized language adapters (LAs) for individual languages. In this work, we show that it is more effective to learn bilingual language pair adapters (BAs) when the goal is to optimize performance for a particular source-target transfer direction. Our novel BAD-X adapter framework trades off some modularity of dedicated LAs for improved transfer performance: we demonstrate consistent gains in three standard downstream tasks, and for the majority of evaluated low-resource languages.
Anthology ID:
2022.naacl-main.130
Volume:
Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies
Month:
July
Year:
2022
Address:
Seattle, United States
Venue:
NAACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
1791–1799
Language:
URL:
https://aclanthology.org/2022.naacl-main.130
DOI:
10.18653/v1/2022.naacl-main.130
Bibkey:
Cite (ACL):
Marinela Parović, Goran Glavaš, Ivan Vulić, and Anna Korhonen. 2022. BAD-X: Bilingual Adapters Improve Zero-Shot Cross-Lingual Transfer. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 1791–1799, Seattle, United States. Association for Computational Linguistics.
Cite (Informal):
BAD-X: Bilingual Adapters Improve Zero-Shot Cross-Lingual Transfer (Parović et al., NAACL 2022)
Copy Citation:
PDF:
https://aclanthology.org/2022.naacl-main.130.pdf
Code
 parovicm/badx