ZHR at BAREC Shared Task 2026: Ordinal-Objective Ensembles with Threshold Calibration and Silver-Label Pretraining for Arabic Readability Assessment

Mahmoud Zaher


Abstract
We describe the ZHR systems for the two sentence-level tracks of the BAREC 2026 Shared Task on Arabic readability assessment: Strict, where training is restricted to the BAREC training split, and Open, which allows any public data. Each sentence receives one of 19 ordinal levels, and systems are ranked by quadratic weighted kappa (QWK). Both tracks use one recipe. Arabic encoders are fine-tuned under five ordinal objectives and averaged with weights from greedy selection, and the averaged score is discretized by 18 thresholds fitted to maximize QWK; every fitted quantity uses the development split. The official Strict system, an ensemble of ten train-split models, scores 85.0 QWK on the blind set. The official Open system adds pretraining on the silver-labeled BAREC-10M corpus and restricts selection to members whose early stopping never touched the calibration split; it scores 85.2. Both clear the official baseline (82.5) by 2.5 and 2.7 QWK while trailing it in exact accuracy, a trade-off that comes from the metric itself and that we measure directly. Negative results are part of the record: a catalog of 19 changes that did not help, each measured under a held-out protocol.
Anthology ID:
2026.arabicnlp-sharedtasks.24
Volume:
Proceedings of The Fourth Arabic Natural Language Processing Conference: Shared Tasks
Month:
October
Year:
2026
Address:
Budapest, Hungary
Editors:
Muhammad Abdul-Mageed, Tamer Elsayed, Houda Bouamor, Samhaa El-Beltagy, Hend Al-Khalifa, Ahmed Abdelali, AbdelRahim Elmadany, Abdellah El Mekki, Kareem Darwish, Sakhar Alkhereyf, Hamzah Luqman, Salam Khalifa, Samar Magdy, Zaid Alyafeai, Injy Hamed, Wajdi Zaghouani, Imed Zitouni
Venue:
ArabicNLP
SIG:
SIGARAB
Publisher:
Association for Computational Linguistics
Note:
Pages:
209–218
Language:
URL:
https://aclanthology.org/2026.arabicnlp-sharedtasks.24/
DOI:
Bibkey:
Cite (ACL):
Mahmoud Zaher. 2026. ZHR at BAREC Shared Task 2026: Ordinal-Objective Ensembles with Threshold Calibration and Silver-Label Pretraining for Arabic Readability Assessment. In Proceedings of The Fourth Arabic Natural Language Processing Conference: Shared Tasks, pages 209–218, Budapest, Hungary. Association for Computational Linguistics.
Cite (Informal):
ZHR at BAREC Shared Task 2026: Ordinal-Objective Ensembles with Threshold Calibration and Silver-Label Pretraining for Arabic Readability Assessment (Zaher, ArabicNLP 2026)
Copy Citation:
PDF:
https://aclanthology.org/2026.arabicnlp-sharedtasks.24.pdf