Weakly and Semi-Supervised Learning for Arabic Text Classification using Monodialectal Language Models

Reem AlYami, Rabah Al-Zaidy


Abstract
The lack of resources such as annotated datasets and tools for low-resource languages is a significant obstacle to the advancement of Natural Language Processing (NLP) applications targeting users who speak these languages. Although learning techniques such as semi-supervised and weakly supervised learning are effective in text classification cases where annotated data is limited, they are still not widely investigated in many languages due to the sparsity of data altogether, both labeled and unlabeled. In this study, we deploy both weakly, and semi-supervised learning approaches for text classification in low-resource languages and address the underlying limitations that can hinder the effectiveness of these techniques. To that end, we propose a suite of language-agnostic techniques for large-scale data collection, automatic data annotation, and language model training in scenarios where resources are scarce. Specifically, we propose a novel data collection pipeline for under-represented languages, or dialects, that is language and task agnostic and of sufficient size for training a language model capable of achieving competitive results on common NLP tasks, as our experiments show. The models will be shared with the research community.
Anthology ID:
2022.wanlp-1.24
Volume:
Proceedings of the Seventh Arabic Natural Language Processing Workshop (WANLP)
Month:
December
Year:
2022
Address:
Abu Dhabi, United Arab Emirates (Hybrid)
Editors:
Houda Bouamor, Hend Al-Khalifa, Kareem Darwish, Owen Rambow, Fethi Bougares, Ahmed Abdelali, Nadi Tomeh, Salam Khalifa, Wajdi Zaghouani
Venue:
WANLP
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
260–272
Language:
URL:
https://aclanthology.org/2022.wanlp-1.24
DOI:
10.18653/v1/2022.wanlp-1.24
Bibkey:
Cite (ACL):
Reem AlYami and Rabah Al-Zaidy. 2022. Weakly and Semi-Supervised Learning for Arabic Text Classification using Monodialectal Language Models. In Proceedings of the Seventh Arabic Natural Language Processing Workshop (WANLP), pages 260–272, Abu Dhabi, United Arab Emirates (Hybrid). Association for Computational Linguistics.
Cite (Informal):
Weakly and Semi-Supervised Learning for Arabic Text Classification using Monodialectal Language Models (AlYami & Al-Zaidy, WANLP 2022)
Copy Citation:
PDF:
https://aclanthology.org/2022.wanlp-1.24.pdf