MOSKA-NLP at AdabEval 2026: Feature-Enriched Ensembling for Arabic Politeness Detection

Nina A. Andriyanova-Almaamary


Abstract
In this paper, we present our system for subtask A of the AdabEval 2026 shared task, which focuses on classifying Arabic text into Polite, Neutral, and Impolite categories. Politeness detection is challenging because it cannot be inferred from lexical meaning alone. This is prominent in Arabic language, where politeness is often conveyed through formulaic expressions, stylistic cues, and dialectal variations. Our approach follows a three-stage strategy. First, we evaluate five Arabic sentence embedding models based on different pretrained encoders to identify a strong representation backbone. Second, we enrich sentence embeddings with explicit lexical, surface-level, and auxiliary signals derived from external models, including dialect, intent, and sarcasm classifiers. Third, we combine predictions from independently trained models, using weighted probability-level ensembling with class-specific decision thresholds to address class imbalance. Experimental results show that feature-enriched representations consistently outperform embedding-only baselines, with additional gains obtained from calibrated ensembling. The proposed system achieves a macro-F1 score of 0.87 and an accuracy of 93% on the official AdabEval 2026 evaluation for subtask A.
Anthology ID:
2026.osact-1.19
Volume:
The 7th Workshop on Open-Source Arabic Corpora and Processing Tools (OSACT7) with 5 Shared Tasks
Month:
May
Year:
2026
Address:
Palma, Mallorca (Spain)
Editors:
Hend Al-Khalifa, Mo El-Haj, Saad Ezzini
Venues:
OSACT | WS
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
165–173
Language:
External URL:
https://lrec.elra.info/lrec2026-ws-osact-19
DOI:
10.63317/56vfmbyh3fsv
Bibkey:
Cite (ACL):
Nina A. Andriyanova-Almaamary. 2026. MOSKA-NLP at AdabEval 2026: Feature-Enriched Ensembling for Arabic Politeness Detection. In The 7th Workshop on Open-Source Arabic Corpora and Processing Tools (OSACT7) with 5 Shared Tasks, pages 165–173, Palma, Mallorca (Spain). Association for Computational Linguistics.
Cite (Informal):
MOSKA-NLP at AdabEval 2026: Feature-Enriched Ensembling for Arabic Politeness Detection (Andriyanova-Almaamary, OSACT 2026)
Copy Citation: