PTUK-NLP at the DialectSentEval 2026 Shared Task: MARBERT-Based Sentiment Classification, Missing-Dialect Effects, and Error Analysis

Tasneem Duridi, Balsam Khaleel, Asma Murad Mallak, Areej Jaber, Elaa Jaber


Abstract
We present our system for Subtask 1 (three-class sentiment classification) of the DialectSentEval 2026 shared task, which evaluates five Arabic dialects using the Multi-Dialect-Sent (MDS-3) hotel-review corpus. Our approach fine-tunes MARBERT with a stratified train/validation split and class-weighted cross-entropy loss. On the official test set, our system achieves an F1 score of 0.8175 and an accuracy of 0.8419, ranking 3rd on the official leaderboard. We identify a substantial dialect mismatch in the dataset: the training set contains no Lebanese reviews, while approximately 20% of the test set is Lebanese. Adding additional Lebanese reviews from OCLAR partially reduces this gap, but does not eliminate it. The error analysis also shows that misclassifications are concentrated in the neutral class and in reviews that express mixed feelings, often with high confidence. These findings highlight the challenges of cross-dialect sentiment classification under unseen-dialect conditions.
Anthology ID:
2026.arabicnlp-sharedtasks.180
Volume:
Proceedings of The Fourth Arabic Natural Language Processing Conference: Shared Tasks
Month:
October
Year:
2026
Address:
Budapest, Hungary
Editors:
Muhammad Abdul-Mageed, Tamer Elsayed, Houda Bouamor, Samhaa El-Beltagy, Hend Al-Khalifa, Ahmed Abdelali, AbdelRahim Elmadany, Abdellah El Mekki, Kareem Darwish, Sakhar Alkhereyf, Hamzah Luqman, Salam Khalifa, Samar Magdy, Zaid Alyafeai, Injy Hamed, Wajdi Zaghouani, Imed Zitouni
Venue:
ArabicNLP
SIG:
SIGARAB
Publisher:
Association for Computational Linguistics
Note:
Pages:
1426–1430
Language:
URL:
https://aclanthology.org/2026.arabicnlp-sharedtasks.180/
DOI:
Bibkey:
Cite (ACL):
Tasneem Duridi, Balsam Khaleel, Asma Murad Mallak, Areej Jaber, and Elaa Jaber. 2026. PTUK-NLP at the DialectSentEval 2026 Shared Task: MARBERT-Based Sentiment Classification, Missing-Dialect Effects, and Error Analysis. In Proceedings of The Fourth Arabic Natural Language Processing Conference: Shared Tasks, pages 1426–1430, Budapest, Hungary. Association for Computational Linguistics.
Cite (Informal):
PTUK-NLP at the DialectSentEval 2026 Shared Task: MARBERT-Based Sentiment Classification, Missing-Dialect Effects, and Error Analysis (Duridi et al., ArabicNLP 2026)
Copy Citation:
PDF:
https://aclanthology.org/2026.arabicnlp-sharedtasks.180.pdf