Northwestern at DialectSentEval 2026: A Comparative Study of MARBERTv2 and AraBERT for Arabic Dialect Sentiment Analysis

Eman Sedqy Shlkamy, Wajdi Zaghouani


Abstract
This paper describes our submission to Subtask 1 of DialectSentEval 2026, which focuses on three class sentiment classification for Arabic dialectal text. The task aims to classify sentences written in five Arabic dialects into positive, negative, or neutral sentiment categories. We investigate the effectiveness of transformer-based Arabic language models through a series of controlled experiments. Specifically, we compare MARBERTv2 and AraBERT while maintaining identical pre processing, training, and evaluation settings to ensure a fair comparison. In addition, we examine whether explicitly providing dialect information as part of the input improves classification performance.

Experimental results on an internal validation split show that MARBERTv2 consistently outperforms AraBERT on this task. Furthermore, removing the explicit dialect tag from the input slightly improves performance, suggesting that MARBERTv2 can effectively capture dialect-specific linguistic patterns directly from the sentence. Our best configuration, which uses MARBERTv2 with sentence-only input, achieves a Macro F1-score of 0.9394 and an accuracy of 94.24%. These findings demonstrate the effectiveness of dialect-aware pre-trained language models for Arabic sentiment analysis and establish a strong baseline for the official shared task evaluation.
Anthology ID:
2026.arabicnlp-sharedtasks.177
Volume:
Proceedings of The Fourth Arabic Natural Language Processing Conference: Shared Tasks
Month:
October
Year:
2026
Address:
Budapest, Hungary
Editors:
Muhammad Abdul-Mageed, Tamer Elsayed, Houda Bouamor, Samhaa El-Beltagy, Hend Al-Khalifa, Ahmed Abdelali, AbdelRahim Elmadany, Abdellah El Mekki, Kareem Darwish, Sakhar Alkhereyf, Hamzah Luqman, Salam Khalifa, Samar Magdy, Zaid Alyafeai, Injy Hamed, Wajdi Zaghouani, Imed Zitouni
Venue:
ArabicNLP
SIG:
SIGARAB
Publisher:
Association for Computational Linguistics
Note:
Pages:
1409–1413
Language:
URL:
https://aclanthology.org/2026.arabicnlp-sharedtasks.177/
DOI:
Bibkey:
Cite (ACL):
Eman Sedqy Shlkamy and Wajdi Zaghouani. 2026. Northwestern at DialectSentEval 2026: A Comparative Study of MARBERTv2 and AraBERT for Arabic Dialect Sentiment Analysis. In Proceedings of The Fourth Arabic Natural Language Processing Conference: Shared Tasks, pages 1409–1413, Budapest, Hungary. Association for Computational Linguistics.
Cite (Informal):
Northwestern at DialectSentEval 2026: A Comparative Study of MARBERTv2 and AraBERT for Arabic Dialect Sentiment Analysis (Shlkamy & Zaghouani, ArabicNLP 2026)
Copy Citation:
PDF:
https://aclanthology.org/2026.arabicnlp-sharedtasks.177.pdf