Lahja at DialectSentEval Shared Task Subtask-1: Seed-Ensembled MARBERT for Multi-Dialect Arabic Sentiment Analysis with Zero-Shot Lebanese Transfer

Shaden Saad AlZeer, Raja Naeem Akram


Abstract
Arabic sentiment analysis must cope with dialects that differ in vocabulary, morphology and spelling. Subtask 1 of the DialectSentEval 2026 shared task poses this problem directly: systems label hotel review sentences from five Arabic dialects as positive, negative or neutral, and one dialect (Lebanese) appears only in the test set. In this paper, we describe the Lahja submission. Since the training corpus is small (1,731 sentences) and consists of parallel translations, we treat unreliable measurement (i.e. leaky validation splits and single training runs) rather than model capacity as the main risk. We fine-tune MARBERTv2 under three random seeds, average their softmax outputs, and validate on a split grouped by review identifier so that all dialect versions of a review stay on the same side. The ensemble reached 0.790 macro-F1 on the official test set, ranking 10th of 12 teams, and lost only about two points from validation to test despite the unseen dialect. Our ablations show that the variation between seeds (about 1.5 points) exceeds the effect of orthographic normalisation or class weighting, and that the normalised ensemble, although best on validation, generalised worst on test. We therefore recommend grouped validation and seed-spread reporting as standard practice for small-data shared tasks.
Anthology ID:
2026.arabicnlp-sharedtasks.184
Volume:
Proceedings of The Fourth Arabic Natural Language Processing Conference: Shared Tasks
Month:
October
Year:
2026
Address:
Budapest, Hungary
Editors:
Muhammad Abdul-Mageed, Tamer Elsayed, Houda Bouamor, Samhaa El-Beltagy, Hend Al-Khalifa, Ahmed Abdelali, AbdelRahim Elmadany, Abdellah El Mekki, Kareem Darwish, Sakhar Alkhereyf, Hamzah Luqman, Salam Khalifa, Samar Magdy, Zaid Alyafeai, Injy Hamed, Wajdi Zaghouani, Imed Zitouni
Venue:
ArabicNLP
SIG:
SIGARAB
Publisher:
Association for Computational Linguistics
Note:
Pages:
1446–1451
Language:
URL:
https://aclanthology.org/2026.arabicnlp-sharedtasks.184/
DOI:
Bibkey:
Cite (ACL):
Shaden Saad AlZeer and Raja Naeem Akram. 2026. Lahja at DialectSentEval Shared Task Subtask-1: Seed-Ensembled MARBERT for Multi-Dialect Arabic Sentiment Analysis with Zero-Shot Lebanese Transfer. In Proceedings of The Fourth Arabic Natural Language Processing Conference: Shared Tasks, pages 1446–1451, Budapest, Hungary. Association for Computational Linguistics.
Cite (Informal):
Lahja at DialectSentEval Shared Task Subtask-1: Seed-Ensembled MARBERT for Multi-Dialect Arabic Sentiment Analysis with Zero-Shot Lebanese Transfer (AlZeer & Akram, ArabicNLP 2026)
Copy Citation:
PDF:
https://aclanthology.org/2026.arabicnlp-sharedtasks.184.pdf