Knowledge Graphs and Distributional Readability of Arabic Documents

Mohamad Chhaytle, Hadi Hamoud, Sally Choker, Chadi Abou Chakra, Mohamad Ballout, Fadi Zaraket


Abstract
This paper addresses automatic readability assessment of Arabic documents. It compares classical readability formulae, fine-tuned Arabic encoders, zero-shot large language models and a Co-occurrence Graphs for Arabic Readability (CoGAR). CoGAR, interpretable and LLM-free, represents a document as a co-occurrence network over content lemmas and use topology features such as density, clustering, path length and degree entropy to predict readability. Evaluated on the BAREC corpus over five ordinal levels, the method reaches a quadratic weighted kappa (QWK) of 0.756, on par with a sparse 35B-parameter LLM and well above the surface formulas Flesch and OSMAN.

An ablation shows that the signal lies in the graph’s shape rather than size, and that alternative graphs based on relations between named-entity carry almost no readability difficulty signal. We further distill a frontier LLM (Gemini) into a small Arabic encoder (NeoAraBERT) by silver labelling 30k chunks, reaching 0.714 QWK. We show that stacking an LLM with CoGAR improves on the LLM alone, reaching QWK 0.839 with GPT-4.1 and 0.852 with the open-weight Qwen-35B, our best result. The two signals are complementary and the gain does not depend on a proprietary model. Finally, we analyze how eight methods including CoGAR distribute reading levels across seven Arabic text sources ranging from children books to expert research manuscripts showing how the methods align with intuitive difficulty.
Anthology ID:
2026.arabicnlp-main.35
Volume:
Proceedings of The Fourth Arabic Natural Language Processing Conference
Month:
October
Year:
2026
Address:
Budapest, Hungary
Editors:
Muhammad Abdul-Mageed, Tamer Elsayed, Houda Bouamor, Samhaa El-Beltagy, Hend Al-Khalifa, Ahmed Abdelali, AbdelRahim Elmadany, Abdellah El Mekki, Kareem Darwish, Sakhar Alkhereyf, Hamzah Luqman, Salam Khalifa, Samar Magdy, Zaid Alyafeai, Injy Hamed, Wajdi Zaghouani, Imed Zitouni
Venue:
ArabicNLP
SIG:
SIGARAB
Publisher:
Association for Computational Linguistics
Note:
Pages:
508–521
Language:
URL:
https://aclanthology.org/2026.arabicnlp-main.35/
DOI:
Bibkey:
Cite (ACL):
Mohamad Chhaytle, Hadi Hamoud, Sally Choker, Chadi Abou Chakra, Mohamad Ballout, and Fadi Zaraket. 2026. Knowledge Graphs and Distributional Readability of Arabic Documents. In Proceedings of The Fourth Arabic Natural Language Processing Conference, pages 508–521, Budapest, Hungary. Association for Computational Linguistics.
Cite (Informal):
Knowledge Graphs and Distributional Readability of Arabic Documents (Chhaytle et al., ArabicNLP 2026)
Copy Citation:
PDF:
https://aclanthology.org/2026.arabicnlp-main.35.pdf