Detecting Languages Unintelligible to Multilingual Models through Local Structure Probes

Louis Clouatre, Prasanna Parthasarathi, Amal Zouaq, Sarath Chandar


Abstract
Providing better language tools for low-resource and endangered languages is imperative for equitable growth.Recent progress with massively multilingual pretrained models has proven surprisingly effective at performing zero-shot transfer to a wide variety of languages.However, this transfer is not universal, with many languages not currently understood by multilingual approaches.It is estimated that only 72 languages possess a “small set of labeled datasets” on which we could test a model’s performance, the vast majority of languages not having the resources available to simply evaluate performances on.In this work, we attempt to clarify which languages do and do not currently benefit from such transfer.To that end, we develop a general approach that requires only unlabelled text to detect which languages are not well understood by a cross-lingual model.Our approach is derived from the hypothesis that if a model’s understanding is insensitive to perturbations to text in a language, it is likely to have a limited understanding of that language.We construct a cross-lingual sentence similarity task to evaluate our approach empirically on 350, primarily low-resource, languages.
Anthology ID:
2022.findings-emnlp.393
Volume:
Findings of the Association for Computational Linguistics: EMNLP 2022
Month:
December
Year:
2022
Address:
Abu Dhabi, United Arab Emirates
Editors:
Yoav Goldberg, Zornitsa Kozareva, Yue Zhang
Venue:
Findings
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
5375–5396
Language:
URL:
https://aclanthology.org/2022.findings-emnlp.393
DOI:
10.18653/v1/2022.findings-emnlp.393
Bibkey:
Cite (ACL):
Louis Clouatre, Prasanna Parthasarathi, Amal Zouaq, and Sarath Chandar. 2022. Detecting Languages Unintelligible to Multilingual Models through Local Structure Probes. In Findings of the Association for Computational Linguistics: EMNLP 2022, pages 5375–5396, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics.
Cite (Informal):
Detecting Languages Unintelligible to Multilingual Models through Local Structure Probes (Clouatre et al., Findings 2022)
Copy Citation:
PDF:
https://aclanthology.org/2022.findings-emnlp.393.pdf