Rawan Nasser Almatham

Author directory

2026

We introduce KASHAF, to our knowledge the first interpretable Semantic Textual Similarity (iSTS) resource and end-to-end system for Arabic: a corpus of 7,811 human-annotated sentence pairs (inter-annotator κ=0.83κ= 0.83 κ=0.83 on a doubly-annotated subset) under a three-level hierarchical schema (sentence, phrase, and word), together with a modular pipeline that produces this layered analysis plus a composite similarity score, the KASHAF Hierarchical Similarity Score (KHSS), calibrated to the ordinal SEM scale. On a held-out cross-batch test split (1,002 pairs, disjoint from training and dev), KASHAF reaches macro-F1 =0.659= 0.659 =0.659 (±0.016±0.016 ±0.016, 5 seeds) and Spearman ρ=0.6715ρ= 0.6715 ρ=0.6715; a rule-based antonym/negation polarity gate provides an empirical score-capping rule for high-similarity contradictions, and the MARBERTv2 backbone leads three alternative Arabic encoders. As an off-the-shelf diagnostic rather than a fine-tuned comparison, four contemporary LLMs under matched prompting reach at most macro-F1 0.215 (standard prompting) and 0.336 (chain-of-thought), a 32–44-point gap that reflects systematic over-prediction of SEM-PARA at the expense of the dominant SEM-PARTIAL class. We release the schema, guidelines, code, models, and a reproducibility pipeline; the corpus is available from KSAA on request.
Academic paraphrasing in Modern Standard Arabic must change expression without distorting scientific content. We study this balance through AraScholar, a corpus of 20,002 manually written source–paraphrase pairs with 522 raw domain labels and item-level provenance. On 2,000 held-out sentences, fine-tuned AraT5v2 favors lexical reuse under beam search, while frontier models produce more extensive rewrites. With corrected Arabic punctuation normalization, diverse decoding and preservation-constrained reranking attain 74.25% coverage of a fixed novelty–preservation band, compared with 52.60% for zero-shot GPT-4o. Gold coverage is only 46.65%, showing that this band measures constraint compliance rather than human quality. Post-hoc target matching substantially improves a calibrated GPT-4o control and narrows the gap, although candidate budgets differ. Crucially, an independent 200-instance audit favors frontier models on meaning and paraphrase quality. We document the corpus, decoding controls, negative distillation result, and metric sensitivity, including test-derived calibration and checkpoint-selection limitations. The findings establish operating-point control as a useful capability while demonstrating why calibration must be evaluated alongside independent assessments of writing quality.