Large language models (LLMs) are increasingly evaluated on legal reasoning tasks, but most benchmarks remain English-centered and test each legal problem in only one linguistic form. We examine whether LLMs preserve legal reasoning performance when the same legal question is expressed across English, formal Arabic, simplified Arabic, and Egyptian Arabic. We construct a controlled benchmark of 200 bar-exam-style multiple-choice questions, preserving the legal facts, answer choices, and correct answer across all language versions. We evaluate GPT-4o, Gemini 2.5 Pro, Grok, and Qwen against a shared human-verified answer key. All models achieve their highest accuracy in English, but Arabic performance varies substantially. Qwen scores 97.0% in English and 79.0% in each Arabic condition, while GPT-4o is the most stable model, with 90.0% in English and 80.0–82.0% across Arabic variants. Gemini 2.5 Pro drops from 80.5% in English to 17.5–21.0% in Arabic, and Grok drops from 45.0% to 16.5–26.5%. These results show that strong English legal accuracy does not guarantee robust Arabic legal reasoning. We argue that multilingual legal evaluation should measure not only accuracy, but also English–Arabic gaps and cross-language variation across semantically aligned legal questions.

Automated Essay Scoring (AES) has been studied since the early days of computational methods. While recent advances incorporating large language models (LLMs) have significantly improved performance, most research has focused on English. In this paper, we present the first benchmark of Arabic cross-prompt AES using Arabic-centric LLMs. We evaluate ALLaM 7B, Jais 8B, and Fanar 9B under zero-shot, few-shot, and fine-tuning settings, and compare them against established AES systems and frontier LLMs. Fine-tuning substantially improves performance across all models, with Fanar achieving state-of-the-art results and outperforming the strongest baseline by 8 points on LAILA dataset. We further show that fine-tuned Arabic-centric LLMs generalize effectively in cross-dataset setting (LAILA → ZAEBUC). Finally, we analyze the best-performing model, examining error patterns, trait interactions, and the effects of multi-trait autoregressive scoring.

Relevance is a central component of Automated Essay Scoring (AES), as it measures how well a student essay addresses a given writing prompt. In this work, we propose AESER, a novel approach to address the problem of graded relevance scoring in Arabic cross-prompt AES. AESER adapts the given prompt more towards the writing space of essays and computes token-level prompt-essay similarity matrices from contextual embeddings. These matrices are processed by a convolutional neural network to predict the relevance score. Experiments on a large Arabic AES dataset show that AESER outperforms strong baselines, establishing a new state-of-the-art performance.

Arabic harmful-language detection has received considerable attention, yet Arabic text detoxification remains underexplored. We introduce AraDetox, a multi-dialect Arabic detoxification dataset comprising 10,500 harmful social-media posts and 84,000 detoxified rewrites generated using GPT-5 and Gemini 2.5 Flash across Modern Standard Arabic, Gulf, Levantine, and Egyptian Arabic. The generated outputs were assessed through human evaluation and automatic analyses of lexical change, semantic preservation, sentiment, and dialectal style. Results show that detoxification is primarily a meaning-preserving rewriting task: substantial lexical and structural reformulation is accompanied by consistently high semantic similarity. Human evaluation confirms successful harmful-language removal while largely preserving the original meaning. Dialectal analyses further indicate that the generated variants exhibit measurable stylistic alignment with reference Arabic dialect corpora. Comparison with existing resources highlights two complementary approaches to detoxification: minimal-edit lexical substitution and meaning-preserving reformulation. Our findings demonstrate that large-scale Arabic detoxification resources can be constructed through LLM-assisted generation and human verification. The dataset is publicly available at https://github.com/ArabicNLP-UK/AraDetox to support future research on Arabic detoxification, safe text generation, and multi-dialect Arabic NLP.

Previous work on Egyptian Arabic in NLP has focused largely on the prestigious Cairene Egyptian Arabic (CEA) dialect, resulting in a lack of representation for the less prestigious Sa’idi Egyptian Arabic (SEA) dialect both in LLM and resource development. Does this lack of representation influence an LLM’s view of the acceptability of SEA (upstream), and does an upstream bias against SEA lead to worse performance (downstream)? We investigate the upstream effect of SEA dialectal features on LLM preferences in a Targeted Syntactic Evaluation (TSE) task which reveals a significant bias against SEA across multiple LLMs. We then analyze the effect of these same features on downstream model performance on MMLU benchmarks and show that models frequently experience a degradation in performance when presented with SEA. This work highlights the need for further exploration on how sub-dialectal variation impacts language technologies.

Arabic-speaking citizens in Israel routinely encounter government website documents written in Modern Standard Arabic (MSA), a variety that differs substantially from the Levantine dialect they use daily. This diglossia gap limits access to public services and creates informational inequity, especially for elderly citizens unfamiliar with modern language tools. We address this problem through three contributions. First, we present DAASI (Disglossia-Aware Arabic Simplification for Inclusion): a new parallel corpus of drawn from Israeli National Insurance Institute documents, constructed via GPT-4o draft simplification followed by expert human validation. Second, we conduct a sociolinguistic survey of 50 Arabic-speaking adults in Israel confirming that government documents are harder to comprehend than school-level texts. Third, we benchmark two Arabic transformer models and GPT-4o across multiple fine-tuning strategies and conduct human evaluations to assess simplified text. Our results confirm that existing Arabic NLP benchmarks do not generalize to the Israeli government domain and that DAASI fills a critical resource gap for inclusive Arabic NLP in a diglossic context.

Arabic gender–number morpho-syntactic agreement exhibits systematic variation beyond functional gender, number, and rationality (GNR) (Alkuhlani and Habash, 2011). We identify and classify agreement patterns involving plural-to-unit and unit-to-members shifts, and introduce _aggregation_ (AGG), a lexeme-level feature and corpus-derived lexical-semantic resource that captures these extended agreement affordances. We extend the GNR agreement framework with aggregation and evaluate it on over one million agreement pairs from ATB1/2/3 and BAREC. AGG improves agreement prediction over a strong baseline by 1.4% absolute on average (38% error reduction). Remaining errors largely reflect analyzer, parser, and treebank limitations. Interpretable agreement assessment of this kind can support the development of pedagogical applications and benchmarking the grammaticality of generative model outputs.[Data and Code are publicly available: https://github.com/CAMeL-Lab/Arabic_GNRA_Lexicon.]

Levantine Arabic (LA) is spoken by tens of millions of people, creating a pressing need for shared benchmarks to evaluate LA speech-language technologies. Evaluating such technology is particularly challenging given LA’s internal diversity and its opaque and non-standardized orthography. We present SHAMS (SHami Annotated Multi-dialect Speech), a benchmark comprising 1,300 utterances drawn from open audio corpora, balanced across five LA varieties (Urban and Rural Palestinian, and Urban Jordanian, Lebanese, and Syrian). Each utterance is represented across four aligned tiers: audio, unvocalized orthography, diacritized text, and phonetic transcription. This structure supports evaluation of various downstream tasks such as diacritization, grapheme-to-phoneme conversion, automatic speech recognition, and audio-to-phoneme, grounded in audio and stratified by variety. We benchmark open and proprietary models across these tasks to demonstrate the utility of this benchmark for measuring progress across LA. We release SHAMS at https://shams-nlp.github.io.

We present BAREC-TB, the first dependency treebank of Arabic children’s literature and educational texts. The 156K-word corpus includes texts drawn from eight sources, including curricula, reading materials, children’s stories, poems, and songs. The treebank is annotated according to the Columbia Arabic Treebank (CATiB) dependency representation. We compare BAREC-TB with existing Arabic treebanks, including the Penn Arabic Treebank and Camel Treebank, using corpus statistics, dependency-based measures of syntactic complexity, readability and vocabulary-related features. We also establish parsing benchmarks using both a trainable parser and large language models. The corpus will be made publicly available to support Arabic NLP, educational NLP, and language pedagogy research: https://barec.camel-lab.com/.

Large language models (LLMs) can generate fluent Arabic answers, yet factual errors remain difficult to detect, localize, explain, and verify. Existing hallucination benchmarks often provide only response-level labels, offering limited support for identifying the exact erroneous content, explaining why it is incorrect, or recovering the correct factual answer. We introduce HalluTruthQA, a fine-grained benchmark for hallucination evaluation in Arabic question answering. HalluTruthQA contains 2,400 expert-curated examples across four knowledge-intensive domains: Islamic knowledge, history, science, and geography. Each example includes an Arabic question, a model-generated answer, a verified reference answer, a binary hallucination label, and six candidate answers for factual verification. For hallucinated answers, the benchmark additionally provides character-level erroneous spans, human-written explanations, and macro- and micro-level hallucination types. We evaluate four open-source LLMs—ALLaM-7B, Falcon-H1R-7B, Qwen3-32B, and SILMA—in a zero-shot setting across four tasks: hallucination detection, span-level localization, factual verification, and explanation evaluation. The results show that these tasks measure complementary capabilities, with no single model achieving the best performance across all of them. The best observed scores are 0.880 Macro-F1 for hallucination detection, 0.516 F1-Sp for span localization, 0.852 LO-Score for factual verification, and 0.644 for explanation evaluation. Our findings highlight the importance of moving beyond binary hallucination detection toward fine-grained evaluation that also localizes, verifies, categorizes, and explains factual errors. Code, data, prompts, and evaluation scripts are available at https://gitlab.com/nlpresearcher/HalluTruthQA

Quranic Arabic poses important challenges for Arabic NLP due to its distinctive morphology, syntax, and orthography. We present MAQAM, a richly annotated Morphosyntactic Arabic Quranic Analysis and Modeling resource covering the entire Quran, providing 22 lexical, inflectional, and clitic features for every word. The resource is constructed through a semi-automatic pipeline combining feature enrichment, morphological synchronization, and expert manual annotation and validation. We further evaluate Quranic morphosyntactic disambiguation under a realistic setting where the input may be represented in either the Uthmani or Imla’i script, and investigate cross-domain transfer between Quranic Arabic and MSA. Our results show that MSA and Quranic resources are complementary for Quranic disambiguation, with the best performance obtained by combining MSA training data, Quran-specific analyzers, and both script representations, whereas for MSA, only integrating Quranic resources provide modest gains. We make MAQAM publicly available.

This paper presents Qwen-MDD-AR, our system for the Iqra’Eval.2 Interspeech 2026 challenge on Modern Standard Arabic pronunciation assessment. Built on Qwen3-ASR-1.7B, it frames Arabic mispronunciation detection and diagnosis as autoregressive speech-to-phoneme generation. We add the 68-phoneme MSA inventory as special tokens, freeze the audio encoder, and adapt the decoder with LoRA. Using only provided challenge data, our submission ranked 6th of 19 teams on the hidden QuranMB.v2 test set, with 0.6702 F1 and 0.0445 PER. In a controlled comparison on our internal mixed held-out set, a Wav2Vec2-BERT CTC baseline trained on the same data achieved better PER, precision, and F1, while Qwen-MDD-AR obtained the same recall and a small Correct Diagnosis advantage (0.9496 vs. 0.9328). Results suggest LoRA-adapted LALMs are a practical alternative for Arabic MDD, while alignment accuracy and inference cost remain open.

Large language models (LLMs) can solve mathematical problems through prompting, yet not all reasoning is cost-justified when using paid APIs. We study prompt-only methods for overthinking control in the black-box setting across four datasets: Arabic GSM8K, English GSM8K, English SVAMP, and ArMATH, each evaluated on 100-question subsets divided into Easy, Medium, and Hard categories. We test Direct Answer, Chain-of-Thought (CoT), First Answer Anchoring (FAA-lite), Confidence-Triggered Escalation (CTE), Self-Consistency, Verification, Arabic prompt variations, and Language-Optimal Prompt Routing (LOPR). CTE achieves 90% accuracy on Arabic GSM8K at 1.14 calls/query, 96% on English GSM8K at 1.06 calls/query, and 98% on SVAMP at 1.08 calls/query. Self-Consistency leads on ArMATH at 90% but costs 2.00 calls/query. Verification consistently increases token cost without accuracy gains. Format-Controlled CoT is the best single-call Arabic method; Bilingual-Glossary CoT is consistently harmful. CTE calibration reveals that explicit low-confidence outputs are rare (<1%), so CTE succeeds mainly because high-confidence predictions are accurate, not through active uncertainty detection.

Classical Arabic poetry generation requires simultaneously satisfying semantic, linguistic, and fine-grained prosodic constraints. Existing systems typically control broad poetic attributes but do not jointly model semantic intent, meter subform, and poem length. We present Shaer, a controllable Classical Arabic poetry generation framework jointly conditioned on natural-language descriptions, meter subforms, and target hemistich counts. To support this task, we construct an enriched corpus of 116,032 classical Arabic poems derived from Ashaar, containing normalized meter-subform labels and automatically generated, validated semantic descriptions. We then adapt Yehia-7B using QLoRA-based supervised fine-tuning with a completion-only objective. Our evaluation combines automatic assessment of base-meter conformity, requested-subform adherence, and length control with three LLM judges, blinded human evaluation, and memorization analysis. Shaer achieves 95.17% base-meter accuracy, 91.75% poem-level meter-subform accuracy, and 83.40% exact count accuracy. Relative to its untuned foundation model, these results represent gains of 68.68, 57.77, and 38.93 percentage points, respectively; Shaer also attains the highest base-meter accuracy among all evaluated systems. Multi-LLM evaluation and a blinded human assessment of top-ranked outputs further indicate competitive semantic and literary quality. Finally, analysis of all 3,481 test generations finds no exact copies from the training corpus or paired source poems. Code, models, and datasets are publicly available.

This paper evaluates whether large language models (LLMs) can demonstrate cultural competence in Egyptian Arabic—not just factual knowledge about culture, but the ability to choose culturally appropriate idiomatic and figurative expressions in everyday social situations. 67 Egyptian participants completed a multiple-choice questionnaire of 90 everyday scenarios to provide human gold labels. Using the same scenarios, each pairing a culturally appropriate Egyptian idiom against a plausible distractor, we evaluated 11 open-source and 3 proprietary models under five prompts and two answer orderings. The results indicated that humans performed near ceiling (98.2%), while LLMs averaged 68.2%. The results also showed that a single accuracy figure conflates three separable behaviors: cultural competence, an answer-position heuristic, and abstention. These behaviors are distributed so unevenly that the model ranking reorders depending on which one is measured. One model’s apparent mid-tier accuracy is almost entirely a “pick the first option” heuristic (a 0.75 normal–reversed gap), and another falls from 51.8% to below chance once refusals to answer are scored as errors. We argue that cultural-competence evaluation should report this decomposition rather than a headline accuracy.

Emirati Arabic remains one of the least-resourced Arabic dialects, with lexical information scattered across corpora, morphological databases, and dictionaries developed under incompatible annotation schemes. We present Gahwa, a machine-readable lexicon constructed by extracting, enriching, and standardizing evidence from an annotated Emirati corpus, a Gulf Arabic verb database, and a Gulf Arabic Wiktionary, unifying their input into one lemma-POS keyed scheme. We evaluate Gahwa through automatic coverage analysis of three Emirati datasets, GumarAE, Mixat, and the Ramsa Corpus, against existing Arabic morphological resources, and through analysis of out-of-vocabulary (OOV) forms. Results show that Gahwa has high word coverage across the three datasets, achieving 89.1-94.2% effective dediacritized lemma-POS token coverage, while the remaining OOV cases primarily reflect lexical gaps, orthographic variations, fused expressions, and morphological gaps.

Arabic large language models must refuse harmful prompts without over-refusing benign or sensitive prompts, yet a single refusal rate hides this trade-off. We evaluate it using benign refusal B and harmful-prompt refusal H, where H measures refusal rather than harmful compliance. Across five Arabic-capable models and 130 runs on the full human-written AraSafe set, refusal-only supervised fine-tuning (SFT) collapses toward blanket refusal, whereas selected mixed-SFT configurations reach H≈90–93% at B=14–23%; four selected configurations exceed H=90% in all three runs, while Fanar does so in two of three. Direct Preference Optimization (DPO) and inference guards change B and H differently across models rather than acting as uniform upgrades. In a blinded 300-response audit, annotator binary-refusal agreement is 89.0% (𝜅=0.78); Qwen3Guard and Aya Expanse 32B reach 88.7% and 91.0% accuracy, respectively, with no conclusive paired difference. Selected SFT raises H on Arabizi for all five models, but none reaches 90%, showing only partial transfer from Modern Standard Arabic; overall, the results support model-specific operating-point selection: set a deployment target and retain only interventions that improve it.

Retrieval-Augmented Generation is used for Islamic question answering, but most systems are evaluated end-to-end, making retrieval failures difficult to isolate from generation failures. We study answer-bearing retrieval for Arabic fiqh, where a passage is relevant only if it states the ruling required by the question. We build a retrieval test collection for Arabic fiqh and use it to evaluate dense, lexical, hybrid, fine-tuned, and madhhab-aware retrieval strategies. The best retriever achieves 0.524 MRR@5, while fine-tuning improves performance to 0.553. Hybrid retrieval provides limited gains for strong models, whereas madhhab-aware filtering more than doubles MRR@5 on school-specific questions. We further present an error analysis showing that the main challenge is distinguishing answer-bearing passages from topically similar passages that do not contain the requested ruling. The best-performing model and fine-tuning dataset are publicly available at https://huggingface.co/collections/somayaeltanbouly/badr-arabic-fiqh-retrieval

Subword tokenization underpins modern neural NLP but can be suboptimal for morphologically rich languages with substantial orthographic variation. In this paper, we compare vocabulary-based and vocabulary-free representations for Arabic-to-English machine translation, including subword, byte-level, and pixel-level approaches. Across 35 evaluation sets spanning Modern Standard Arabic (MSA) and 18 dialects, all approaches achieve comparable translation performance with sufficient supervision. While subword models perform best with limited MSA pretraining data, differences diminish with scale and dialect-specific fine-tuning. Pixel-based models further exhibit the greatest robustness to unseen orthographic variation and provide the highest training and inference throughput. We make our code and models publicly available.

Arabic machine translation benchmarks provide limited coverage of variation within Saudi Arabic. We introduce SAFIR, a Saudi-centric benchmark of 610 unique sentence pairs organized into 12 creator specific evaluation splits across five regional groups: Najdi, Hijazi, Southern, Northern, and Eastern. Native speakers authored the Arabic sentences and their English translations; 235 pairs received independent double review for dialect authenticity, translation adequacy, and fluency. We evaluate twelve systems, comprising losed-weight LLMs, open-weight LLMs, and dedicated MT models, in the zero-shot Saudi→English setting using reference-based metrics and GPT-4o judgments. Closed-weight LLMs obtain the strongest aggregate results. Gemini 2.5 Pro achieves the highest average BLEU (34.41), while GPT-5 achieves the highest BERTScore (83.60). Rankings vary across splits, and Southern varieties present particular challenges in judged adequacy. Differences between reference overlap, adequacy, and fluency support reporting performance by sampled variety and evaluation dimension. SAFIR and its leaderboard support more detailed assessment of Saudi Arabic translation.

We introduce QuranicMMLU, a benchmark for evaluating generative AI on Quranic Arabic across multiple dimensions of linguistic complexity. Existing Quranic benchmarks center on general question answering and semantic retrieval, without probing specific linguistic competencies or stratifying by cognitive demand and verse difficulty. We construct a five-pillar Quranic taxonomy spanning Phonology, Morphology, Syntax, Semantics, and Pragmatics, with 31 leaves covering phenomena from tajwid and root-and-pattern morphology to occasions of revelation and inter-surah coherence. For each leaf we generate questions stratified by Bloom’s cognitive level and verse perplexity, then have LLM as a judge to independently answer and score every item and route the annotations to manual review. The resulting dataset comprises 980 human reviewed questions, each issued in both open ended and multiple-choice form. We benchmark 12 systems on these items and find that the Islamic-specialized model leads, yet every system scores higher on multiple-choice accuracy (average 84%) than open-ended answer quality (average 60%): the two rankings agree closely (Kendall’s τ = 0.73), but multiple-choice scoring hides failures that surface only once answer choices are removed. QuranicMMLU thus offers a rigorous, linguistically grounded framework for evaluating Arabic NLP in the Quranic domain.

Recent transformer-based language models rely on subword tokenization and contextual embeddings, raising questions about the continued effectiveness of traditional preprocessing techniques such as stemming. This issue remains underexplored for low-resource Arabic dialects, particularly Tunisian Dialect (TD). In this paper, we investigate the impact of light and aggressive stemming on two main Tunisian NLP tasks: sentiment analysis and domain classification. We compare classical machine learning methods with transformer-based Arabic language models, including TunBERT. To support this study, we release an open Tunisian stemming resource containing 7,730 words annotated with both light and aggressive stems. Our experiments provide a comprehensive analysis of TD processing and show that the effectiveness of stemming depends on both the task and the model type.

Metaphor research has gained increasing attention due to its relevance to linguistic creativity, language use, cognitive processes, and related areas. While many efforts have been devoted to metaphor identification and annotation in English and other languages, Arabic remains under-resourced in this area. In this work, we propose the Arabic Metaphor Identification Procedure (AraMIP), a novel guideline for Arabic metaphor annotation. AraMIP builds on the widely used Metaphor Identification Procedure Vrije Universiteit (MIPVU) framework, incorporating adaptations that account for the language-specific properties of Arabic. We distinguish three major types of Arabic figurative language: isti‘āra (metaphor), kināya (metonymy/indirect expression), and tashbīh (simile) and annotate a pilot dataset of 300 sentences (5277 words). Our analysis reveals key challenges specific to Arabic, including morphological complexity, inconsistencies in dictionary sense ordering, and the absence of standardized contextual materials for annotators. This work contributes a first step toward standardized Arabic figurative instances and facilitates the development of larger annotated resources, thereby supporting future research on figurative language in Arabic.

Academic paraphrasing in Modern Standard Arabic must change expression without distorting scientific content. We study this balance through AraScholar, a corpus of 20,002 manually written source–paraphrase pairs with 522 raw domain labels and item-level provenance. On 2,000 held-out sentences, fine-tuned AraT5v2 favors lexical reuse under beam search, while frontier models produce more extensive rewrites. With corrected Arabic punctuation normalization, diverse decoding and preservation-constrained reranking attain 74.25% coverage of a fixed novelty–preservation band, compared with 52.60% for zero-shot GPT-4o. Gold coverage is only 46.65%, showing that this band measures constraint compliance rather than human quality. Post-hoc target matching substantially improves a calibrated GPT-4o control and narrows the gap, although candidate budgets differ. Crucially, an independent 200-instance audit favors frontier models on meaning and paraphrase quality. We document the corpus, decoding controls, negative distillation result, and metric sensitivity, including test-derived calibration and checkpoint-selection limitations. The findings establish operating-point control as a useful capability while demonstrating why calibration must be evaluated alongside independent assessments of writing quality.

Arabic machine translation remains challenging because Modern Standard Arabic and dialectal varieties such as Egyptian Colloquial Arabic differ substantially in morphology, lexical choice, register, and everyday usage. We introduce a compact Arabic-centric encoder–decoder machine translation model designed to strengthen translation involving Modern Standard Arabic and Egyptian Arabic while preserving multilingual translation ability. We pretrain from scratch with multilingual Seq2Seq training and contrastive cross-lingual alignment, using an Arabic-centered data mixture covering Arabic, English, Egyptian Arabic, and 111 additional languages. We further apply Direct Preference Optimization (DPO) for our Seq2Seq model using grounded preference pairs targeting Arabic-specific translation errors, including literal translation, register mismatch, lexical ambiguity, domain terminology, and dialectal naturalness. We evaluate our model on FLORES, WMT24++, and NTREX using spBLEU. Results show that our model remains competitive with substantially larger LLM baselines and achieves its clearest gains on Arabic-centered directions, especially Egyptian Arabic. DPO improves FLORES translation into English by approximately +2 spBLEU for Egyptian Arabic and +2 spBLEU for Modern Standard Arabic. Our findings support targeted Arabic-centric Seq2Seq modeling as an efficient path for dialect-sensitive machine translation.

Automatic speech recognition for Algerian social-media speech is hard because of dialectal variation, Arabic-French code-switching, and non-standard orthography, yet these difficulties are usually summarised by a single word error rate that hides where recognition fails and whether gains reach information-bearing content. We evaluate seven systems on roughly 30 minutes of Algerian political social-media speech, annotated by language variety as Algerian dialect, Modern Standard Arabic, code-switched, or French: a Whisper-large-v3 baseline, five Whisper variants fine-tuned on different combinations of broadcast, dialectal, and code-switched speech, and the prompted Qwen3-Omni model. The best system reduces overall WER from 60.1% to 31.3%, but still reaches 44.0% on code-switched speech, and named entities remain harder than ordinary tokens, with fewer than half preserved. These results show that ASR difficulty on Algerian social-media speech cannot be read from overall WER alone; it must be measured separately by language variety and by whether named entities survive.

Hateful memes are a growing form of multimodal online harm, where hostile intent is often conveyed through the joint interpretation of images, text, cultural references, and implicit targets. While hateful meme detection has advanced in high-resource languages, Arabic remains underexplored, with existing meme resources focusing mainly on propaganda or coarse harmful-content labels. We introduce AHA-Memes, which is, to our knowledge, the first large Arabic hateful meme benchmark with fine-grained, multi-label annotations. The dataset includes 5K manually annotated memes using a taxonomy that captures hate types, i.e., attack strategies. We further provide ~66K silver-labeled memes to support future studies. We benchmark text-only, image-only, and late-fusion multimodal models, as well as few-shot in-context learning (ICL) and open- and closed-weight Vision-Language Models (VLMs) under zero-shot and fine-tuning settings. Our results establish strong baselines and highlight key challenges in culturally grounded Arabic hateful meme detection. We release the dataset, annotation guidelines, and evaluation scripts to support future research. (**Resources:** https://github.com/MohamedBayan/AHA-Memes) (**Hugging Face:** https://huggingface.co/datasets/QCRI/AHA-MEMES)

Arabic Dialect Identification (ADI) aims at automatically assigning an utterance—written or spoken—to a single variety, be it Modern Standard Arabic (MSA) or a variety of Dialectal Arabic (DA). To complement ADI, the Arabic Level of Dialectness (ALDi) has been proposed to model the different dialect levels that exist between MSA and Highly Dialectal text. However, current ALDi estimation models operate on text inputs only. This paper presents the first step toward extending ALDi to the speech modality. First, we use the automatically estimated ALDi score of a spoken utterance’s transcript as the silver-standard target score. We then compare the performance of two approaches at automatically predicting these silver-standard ALDi scores from spoken utterances: (1) a cascaded approach where automatic speech recognition is used to generate automatic transcripts that are then fed to a text-based ALDi estimation model, and (2) a direct ALDi estimation from speech. We use SADA—a dataset of 435 hours of manually transcribed Arabic speech—as the fine-tuning source data, and evaluate the two approaches on three datasets that cover different varieties of Arabic. Our results indicate that estimating silver-standard ALDi scores directly from speech is on-par or slightly worse than the cascaded approach for in-domain data. However, the direct approach generalizes better, especially for dialects where the fine-tuned ASR backbone of the cascaded approach has relatively large word-error-rates. This paper paves the way towards predicting ALDi scores directly from speech inputs.

We introduce Definition-Guided Learning (DGL), a contrastive training framework that uses dictionary definitions as semantic queries and lemmas as retrieval targets, instantiated here for Arabic reverse dictionary retrieval. We construct a root-stratified Arabic lexical benchmark that limits morphological root overlap between training and test partitions. Zero-shot evaluation suggests a notable gap, with the best pre-trained model achieving only 11.16% Top@1 accuracy on this benchmark. Fine-tuning with DGL yields 41.13% Top@1 and NDCG@10 of 49.32%, an improvement of roughly 3.7 times over the strongest zero-shot baseline we tested. Our results suggest that in-batch negative sampling tends to outperform Triplet Loss in this setting, and that definition length and lexical difficulty appear to trade off in importance across retrieval depth. These findings offer preliminary evidence that lexicographic resources can serve as a useful supervisory signal for Arabic embedding learning, at least within the scope of the benchmark studied here.

Automatic Quranic recitation assessment requires accurate detection of phonetic and Tajweed-related pronunciation errors while providing low-latency feedback. However, most speech recognition models are designed for offline processing and are too large for deployment on consumer mobile devices.

We present a real-time Quranic recitation assessment system based on a multi-head Wav2Vec2-BERT model trained to jointly predict phonemes and Tajweed Sifat attributes using Connectionist Temporal Classification. To enable streaming inference without retraining, we introduce an Adaptive Eager-Emission framework that combines heuristic output commitment, dynamic context expansion, and overlap recovery, reducing perceived latency while preserving long-range temporal features.

For efficient on-device deployment, we propose a structured compression pipeline based on encoder pruning, recovery fine-tuning, and INT8 quantization with ONNX optimization, reducing a 600M-parameter, 2.4 GB model to a 24.5M-parameter, 36 MB model while maintaining recognition performance.

The streaming framework consistently outperforms static sliding-window approaches across the accuracy–latency trade-off, achieving responsiveness as low as 1.0 s at roughly 2.1% phoneme error rate (PER), or near-offline performance below 1% PER at 6.0 s average chunk latency, while the production streaming configuration maintains 99.45% average Sifat attribute accuracy. On a mid-range Android CPU, the compressed model runs at a real-time factor of 0.225 with a peak memory of about 600 MB.

We introduce KASHAF, to our knowledge the first interpretable Semantic Textual Similarity (iSTS) resource and end-to-end system for Arabic: a corpus of 7,811 human-annotated sentence pairs (inter-annotator κ=0.83κ= 0.83 κ=0.83 on a doubly-annotated subset) under a three-level hierarchical schema (sentence, phrase, and word), together with a modular pipeline that produces this layered analysis plus a composite similarity score, the KASHAF Hierarchical Similarity Score (KHSS), calibrated to the ordinal SEM scale. On a held-out cross-batch test split (1,002 pairs, disjoint from training and dev), KASHAF reaches macro-F1 =0.659= 0.659 =0.659 (±0.016±0.016 ±0.016, 5 seeds) and Spearman ρ=0.6715ρ= 0.6715 ρ=0.6715; a rule-based antonym/negation polarity gate provides an empirical score-capping rule for high-similarity contradictions, and the MARBERTv2 backbone leads three alternative Arabic encoders. As an off-the-shelf diagnostic rather than a fine-tuned comparison, four contemporary LLMs under matched prompting reach at most macro-F1 0.215 (standard prompting) and 0.336 (chain-of-thought), a 32–44-point gap that reflects systematic over-prediction of SEM-PARA at the expense of the dominant SEM-PARTIAL class. We release the schema, guidelines, code, models, and a reproducibility pipeline; the corpus is available from KSAA on request.

Automatic machine-translation (MT) metrics collapse heterogeneous errors into a single score. For Arabic, no fine-grained resource exists for diagnosing error types and severity. We introduce QUANTA, a 9,032-instance En to Ar MT-error dataset (738 real UNPC + 7,856 synthetic training instances, both structurally validated, plus a held-out 438-instance real Gold test set) annotated with Mizan (ميزان, "scale"), a new 23-class MQM-compatible taxonomy organised in a four-level hierarchy: 23 leaf classes rolling up to 14 subtypes, 5 parent groups, and 4 TQA dimensions. To make the resource usable, we release the Raqeeb (رقيب, "observer") classifier, an AraBERT-v2 model for predicting Mizan 23-class labels (5-fold Gold macro-F1 = 0.614 ± 0.017). Data quality is supported by ETCA (Error Taxonomy Cross-vendor Audit), an LLM-as-judge check in which a different-vendor model audits QUANTA instances for anchor validity and label fidelity, curbing same-family self-preference bias, and by dual-annotator validation (Gwet’s AC1 = 0.97 synthetic; 0.83 real) on core taxonomy criteria. Ablations show that anchor-guided framing is decisive (without it, macro-F1 collapses to 0.114); that 738 structurally validated real samples are ≈12× more sample-efficient than synthetic data; and that, restricted to the classes with real-tier training support, combining both tiers outperforms real (0.656) or synthetic (0.312) alone. Code, data, prompts, and checkpoints are publicly available.

Spoken Arabic dialect identification (DID) has advanced quickly, but Libyan Arabic remains under-served and, where it appears, is labeled only at the country level, hiding the variation between the Libyan coast and interior. We present a Libyan Arabic speech corpus annotated at the sub-regional level: Tripolitania (west), Cyrenaica (east), and Fezzan (south). It contains 133.7 hours of single-speaker segments from 645 speakers across 161 channels (YouTube and TikTok sources). An automated pipeline (diarization, speaker embedding, clustering, cross-episode linking) produces silver labels from channel provenance. One annotator then reviewed the resulting speaker clusters and confirmed the provenance region for 95.5% of the kept speakers (a within-annotator figure). We release speaker-disjoint splits. An off-the-shelf Arabic DID model recognizes our verified Libyan speech as Libyan only 74% of the time and cannot resolve the region. On the new three-way task, the encoder matters more than the classifier: a probe over speaker-recognition ECAPA embeddings reaches 0.525 macro-F1, over a language-identification ECAPA reaches 0.660, and fine-tuning that encoder reaches 0.702 (75.5% accuracy), far above the 0.20 floor. A channel-disjoint re-test reaches 0.609, versus 0.702 speaker-disjoint; Fezzan is hardest at 0.313. The task is learnable, but Fezzan performance remains low. We release the corpus, cached embeddings, and code.

Arabic NLP faces a fundamental mismatch between training and deployment conditions. While most language models are trained predominantly on Modern Standard Arabic (MSA), real-world communication occurs largely in spoken dialects. To bridge this gap, we leverage social media content from Reddit, Twitter, and YouTube as a scalable proxy for spoken Arabic. We construct a 1.04-billion-token pretraining corpus and train SALM, a compact 336M-parameter decoder-only language model pre-trained via next-token prediction alone, no masked language modelling, no dialect labels, no task-specific signal. Despite an order-of-magnitude smaller training corpus than existing dialectal PLMs and a generative training objective that provides no direct classification signal, SALM matches MARBERT, explicitly pre-trained on one billion dialectal tweets, within 0.7 percentage points on MADAR-26 dialect identification accuracy and outperforms it on macro-F1. Under full fine-tuning, SALM surpasses all baselines, achieving 69.1% accuracy on 26-way city-level dialect classification. Under the same frozen probe on binary Arabic sentiment (AJGT), SALM achieves 92.2% accuracy, topping all Arabic PLMs, encoder and decoder alike. We release the corpus and trained model to facilitate future research on Arabic spoken language modelling.

This paper addresses automatic readability assessment of Arabic documents. It compares classical readability formulae, fine-tuned Arabic encoders, zero-shot large language models and a Co-occurrence Graphs for Arabic Readability (CoGAR). CoGAR, interpretable and LLM-free, represents a document as a co-occurrence network over content lemmas and use topology features such as density, clustering, path length and degree entropy to predict readability. Evaluated on the BAREC corpus over five ordinal levels, the method reaches a quadratic weighted kappa (QWK) of 0.756, on par with a sparse 35B-parameter LLM and well above the surface formulas Flesch and OSMAN.

An ablation shows that the signal lies in the graph’s shape rather than size, and that alternative graphs based on relations between named-entity carry almost no readability difficulty signal. We further distill a frontier LLM (Gemini) into a small Arabic encoder (NeoAraBERT) by silver labelling 30k chunks, reaching 0.714 QWK. We show that stacking an LLM with CoGAR improves on the LLM alone, reaching QWK 0.839 with GPT-4.1 and 0.852 with the open-weight Qwen-35B, our best result. The two signals are complementary and the gain does not depend on a proprietary model. Finally, we analyze how eight methods including CoGAR distribute reading levels across seven Arabic text sources ranging from children books to expert research manuscripts showing how the methods align with intuitive difficulty.

Combining LLMs for Arabic punctuation restoration should help when their errors are complementary, but aggregation failure can reflect either missing coverage, where no predictor proposes the correct label, or failed selection, where the correct label is present but not selected. We investigate this distinction with vote-based, text-blind aggregators that use predictor labels, agreement, position, and neighbouring predictions, but not Arabic words. We evaluate three frontier LLMs and a fine-tuned AraBERT encoder on AraPunc-sub and ABC. The LLMs show strongly correlated errors: when all three are wrong, they choose the same incorrect label in 79.4% of cases on AraPunc-sub and 77.8% on ABC. Yet the four-predictor oracle achieves macro-F₁ scores of 0.857 and 0.778, surpassing the best standalone predictors (0.639 and 0.496). Leave-one-predictor-out analysis shows that AraBERT supplies the largest unique oracle coverage. Standard aggregation recovers only part of this headroom, and no tested aggregator outperforms standalone AraBERT on AraPunc-sub. A selectability probe confirms that some covered disagreements are learnable, but a deployable router offers only modest, corpus-dependent gains. Even a text-aware peer-deliberation step deepens rather than breaks the correlation, as weaker models conform to their peers. Hence, in the tested text-blind regime, the bottleneck is primarily selection rather than coverage, without ruling out text-aware selectors.

Automatic Speech Recognition (ASR) systems often underperform for children and non-native speakers, while adapting adult ASR models to child speech can cause adult-speech forgetting. We study child ASR adaptation with adult retention across Arabic and English. We compare full fine-tuning, LoRA, and post-hoc weight-space merging across encoder–decoder, encoder–CTC, and AudioLLM-based ASR systems. Experiments use Arabic native and non-native child speech, English MyST child speech, and adult benchmarks from MGB-2 and LibriSpeech test-clean. We evaluate recognition quality with WER and quantify the adaptation–retention trade-off using Retention Index, Child Adaptation Gain, and Adaptation Recovery. Results show that child adaptation is necessary, especially for non-native Arabic and English child speech, but direct adaptation often reduces adult ASR performance. Bilingual adaptation is more stable than language-specific adaptation. Weight-space merging often improves the trade-off, especially for encoder–CTC, Whisper, and AudioLLM-based ASR, with LERP favoring adult retention and TIES recovering stronger child gains. For the encoder–decoder model, direct bilingual fine-tuning remains strongest in raw WER.