Cascading Adaptors to Leverage English Data to Improve Performance of Question Answering for Low-Resource Languages

Hariom Pandya, Bhavik Ardeshna, Brijesh Bhatt


Abstract
Transformer based architectures have shown notable results on many down streaming tasks including question answering. The availability of data, on the other hand, impedes obtaining legitimate performance for low-resource languages. In this paper, we investigate the applicability of pre-trained multilingual models to improve the performance of question answering in low-resource languages. We tested four combinations of language and task adapters using multilingual transformer architectures on seven languages similar to MLQA dataset. Additionally, we have also proposed zero-shot transfer learning of low-resource question answering using language and task adapters. We observed that stacking the language and the task adapters improves the multilingual transformer models’ performance significantly for low-resource languages. Our code and trained models are available at: https://github.com/CALEDIPQALL/
Anthology ID:
2021.icon-main.66
Volume:
Proceedings of the 18th International Conference on Natural Language Processing (ICON)
Month:
December
Year:
2021
Address:
National Institute of Technology Silchar, Silchar, India
Editors:
Sivaji Bandyopadhyay, Sobha Lalitha Devi, Pushpak Bhattacharyya
Venue:
ICON
SIG:
Publisher:
NLP Association of India (NLPAI)
Note:
Pages:
544–549
Language:
URL:
https://aclanthology.org/2021.icon-main.66
DOI:
Bibkey:
Cite (ACL):
Hariom Pandya, Bhavik Ardeshna, and Brijesh Bhatt. 2021. Cascading Adaptors to Leverage English Data to Improve Performance of Question Answering for Low-Resource Languages. In Proceedings of the 18th International Conference on Natural Language Processing (ICON), pages 544–549, National Institute of Technology Silchar, Silchar, India. NLP Association of India (NLPAI).
Cite (Informal):
Cascading Adaptors to Leverage English Data to Improve Performance of Question Answering for Low-Resource Languages (Pandya et al., ICON 2021)
Copy Citation:
PDF:
https://aclanthology.org/2021.icon-main.66.pdf
Optional supplementary material:
 2021.icon-main.66.OptionalSupplementaryMaterial.pdf
Code
 Bhavik-Ardeshna/Question-Answering-for-Low-Resource-Languages
Data
MLQAXQuAD