Arabic Dialects Identification for All Arabic countries

Ahmed Aliwy, Hawraa Taher, Zena AboAltaheen


Abstract
Arabic dialects are among of three main variant of Arabic language (Classical Arabic, modern standard Arabic and dialectal Arabic). It has many variants according to the country, city (provinces) or town. In this paper, several techniques with multiple algorithms are applied for Arabic dialects identification starting from removing noise till classification task using all Arabic countries as 21 classes. Three types of classifiers (Naïve Bayes, Logistic Regression, and Decision Tree) are combined using voting with two different methodologies. Also clustering technique is used for decreasing the noise that result from the existing of MSA tweets in the data set for training phase. The results of f-measure were 27.17, 41.34 and 52.38 for first methodology without clustering, second methodology without clustering, and second methodology with clustering, the used data set is NADI shared task data set.
Anthology ID:
2020.wanlp-1.32
Volume:
Proceedings of the Fifth Arabic Natural Language Processing Workshop
Month:
December
Year:
2020
Address:
Barcelona, Spain (Online)
Editors:
Imed Zitouni, Muhammad Abdul-Mageed, Houda Bouamor, Fethi Bougares, Mahmoud El-Haj, Nadi Tomeh, Wajdi Zaghouani
Venue:
WANLP
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
302–307
Language:
URL:
https://aclanthology.org/2020.wanlp-1.32
DOI:
Bibkey:
Cite (ACL):
Ahmed Aliwy, Hawraa Taher, and Zena AboAltaheen. 2020. Arabic Dialects Identification for All Arabic countries. In Proceedings of the Fifth Arabic Natural Language Processing Workshop, pages 302–307, Barcelona, Spain (Online). Association for Computational Linguistics.
Cite (Informal):
Arabic Dialects Identification for All Arabic countries (Aliwy et al., WANLP 2020)
Copy Citation:
PDF:
https://aclanthology.org/2020.wanlp-1.32.pdf