Analyzing the Effects of Reasoning Types on Cross-Lingual Transfer Performance

Karthikeyan K, Aalok Sathe, Somak Aditya, Monojit Choudhury


Abstract
Multilingual language models achieve impressive zero-shot accuracies in many languages in complex tasks such as Natural Language Inference (NLI). Examples in NLI (and equivalent complex tasks) often pertain to various types of sub-tasks, requiring different kinds of reasoning. Certain types of reasoning have proven to be more difficult to learn in a monolingual context, and in the crosslingual context, similar observations may shed light on zero-shot transfer efficiency and few-shot sample selection. Hence, to investigate the effects of types of reasoning on transfer performance, we propose a category-annotated multilingual NLI dataset and discuss the challenges to scale monolingual annotations to multiple languages. We statistically observe interesting effects that the confluence of reasoning types and language similarities have on transfer performance.
Anthology ID:
2021.mrl-1.8
Volume:
Proceedings of the 1st Workshop on Multilingual Representation Learning
Month:
November
Year:
2021
Address:
Punta Cana, Dominican Republic
Editors:
Duygu Ataman, Alexandra Birch, Alexis Conneau, Orhan Firat, Sebastian Ruder, Gozde Gul Sahin
Venue:
MRL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
86–95
Language:
URL:
https://aclanthology.org/2021.mrl-1.8
DOI:
10.18653/v1/2021.mrl-1.8
Bibkey:
Cite (ACL):
Karthikeyan K, Aalok Sathe, Somak Aditya, and Monojit Choudhury. 2021. Analyzing the Effects of Reasoning Types on Cross-Lingual Transfer Performance. In Proceedings of the 1st Workshop on Multilingual Representation Learning, pages 86–95, Punta Cana, Dominican Republic. Association for Computational Linguistics.
Cite (Informal):
Analyzing the Effects of Reasoning Types on Cross-Lingual Transfer Performance (K et al., MRL 2021)
Copy Citation:
PDF:
https://aclanthology.org/2021.mrl-1.8.pdf
Code
 microsoft/taxixnli
Data
MultiNLITaxiNLI