Hassan Barmandah

Author directory

2026

This paper presents the Fine-Tashkeel system for Task 2 of the KSAA-2026 Shared Task on Automatic Diacritization of Speech Dictation. Diacritization of speech-derived Arabic text poses challenges due to dialectal variation, morphological ambiguity, and the absence of acoustic cues in text-only pipelines. Our approach treats diacritization as a character-level sequence-to-sequence task, mapping undiacritized text directly to its fully diacritized form. We evaluate 18 models spanning text-only, ASR-augmented, and fine-tuned configurations, finding that text-only Seq2Seq approaches outperform off-the-shelf multimodal models—a gap we attribute to task mismatch in generic ASR systems rather than an inherent audio limitation. Our best submission, using zero-shot inference without task-specific training, achieved a Diacritic Error Rate (DER) of 10.56%, Word Error Rate (WER) of 34.47%, and Sentence Error Rate (SER) of 79.88%, ranking 5th out of 7 teams. Per-nationality error analysis reveals significant dialectal variation (Egyptian 3.70% vs. Algerian 13.73% DER), and diagnostic analysis confirms that case endings and vowel ambiguity are the primary bottlenecks. Code and evaluation scripts are publicly available.
This paper presents Ketaba-OCR-LoRA, a system developed for the NakbaNLP 2026 Shared Task on Arabic Manuscript Understanding (Subtask 2), which targets the transcription of the historically significant Omar Al-Saleh Memoir Collection written in Ruq’ah and Naskh scripts. We propose a parameter-efficient adaptation of a publicly available pretrained Arabic-English Handwritten Text Recognition (HRT) model, originally trained on handwritten corpora including the Muharaf dataset. Instead of adapting general Vision-Language Models from scratch, we fine-tune the HRT backbone using Low-Rank Adaptation (LoRA) and 4-bit quantization (QLoRA), reducing memory requirements from 40GB to approximately 8GB. Our final submission combines multiple model variants through a novel Linear+Boost weighted ensemble strategy. Our approach achieves a CER of 0.0819 and WER of 0.2588 on the blind test set (per-line evaluation), ranking 1st on per-line evaluation; on the official corpus-wide leaderboard, we rank 3rd (CER 0.0938, WER 0.2996). This work demonstrates that specialized pretrained HRT models substantially outperform general-purpose Vision-Language Models for Arabic manuscript transcription, and that parameter-efficient fine-tuning provides a practical and reproducible approach for low-resource cultural heritage digitization.
Stance detection identifies an author’s position toward a specified target, yet Arabic social-media text remains challenging because stance is often implicit and shaped by dialect, sarcasm, informal expression, and culturally specific references. StanceEval-2026 evaluates this problem in two transfer settings: a related target within a seen thematic domain (Track 1) and entirely unseen topics (Track 2). We compare four modeling strategies: single-task Arabic encoder fine-tuning, multi-task learning with sentiment and sarcasm supervision, 4-bit QLoRA adaptation of the ALLaM-7B model, and Retrieval-Augmented In-Context Learning (RAG-ICL) with frozen large language models (LLMs). Our strongest single-backend configuration is a data-centric RAG-ICL pipeline combining label-balanced retrieval, hashtag and negation adaptation, text cleaning, similarity filtering, and target descriptions. A majority-vote ensemble of this configuration across three LLM backends obtains official Favg2 scores of 0.8719 on Track 1 and 0.9016 on Track 2, ranking 3rd and 7th, respectively. Under the shared-task test conditions, the retrieval-based systems substantially outperform the supervised systems evaluated here, particularly on Track 2.
Hierarchical, definition-guided genre classification places a short Arabic text under a broad genre and, within it, a fine-grained specific genre, using only a natural-language definition of each. We present NAMAA’s submission to AraGenre 2026. Fine-tuned bi-encoder classifiers reached 0.93 hierarchical macro F1 on development via data augmentation and cross-lingual pretraining, but collapsed to as low as 0.22 on the released test set, having specialized to a hand-engineered definition style that missed the test set’s larger, 74-genre taxonomy. We root-caused this gap and switched to zero-shot LLM classification, matching the best fine-tuned ensemble’s test score (0.6899 hierarchical F1) with no training; layering self-consistency sampling and disagreement resolution raised this further to 0.7013, ranking 3rd of 18 teams. Specific Macro F1 (0.5114) lags well behind Broad Macro F1 (0.8913), showing that most remaining errors concentrate in fine-grained genre choices rather than broad-category mistakes. Zero-shot LLM prompting generalizes across this definition and taxonomy shift where fine-tuned, definition-conditioned encoders do not. We release our code and system resources to support reproducibility and further research.[<https://github.com/NAMAA-ORG/Aragenre-Definition-Guided-Genre>].
Arabic sentiment swap aims to reverse the polarity of a dialectal sentence while preserving its underlying meaning. We present NAMAA’s system for DialectSentEval 2026 Subtask 2. Fine-tuning AraT5v2 achieves 79.56 development sentiment-style accuracy, while reranked sampling over eight candidates, using sentiment agreement and content-overlap signals, raises this to 93.57. Cross-backbone ensembling further reaches a development score of 96.78. On the hidden test set, however, this ordering does not transfer: a widened rerank pool with a re-tuned content weight achieves our best official score of 0.7597, whereas the most complex all-source ensemble scores 0.7343. We identify two likely contributors to this development-to-test mismatch: the absence of gold source-polarity labels at test time, which requires classifier-based polarity inference, and train–development sentence duplication that may inflate development performance. These results show that additional generation and ensemble complexity can improve development metrics substantially without guaranteeing better generalization under shifted evaluation conditions.
Arabic dialect sentiment analysis is challenging because regional varieties show considerable lexical, morphological, syntactical, and orthographic variation. This paper presents a transformer-based system developed for DialectSentEval 2026 Subtask 1, a three-class sentiment classification task covering Moroccan, Egyptian, Jordanian, and Saudi Arabic. The system independently fine-tunes three pretrained Arabic language models: AraBERTv2, MARBERTv2, and CAMeLBERT, using a common training protocol. The implementation uses an 80/20 stratified train-validation split, a maximum sequence length of 128, a batch size of 16, a learning rate of 2 × 10-5, and focal loss with 𝛾=2=2. Model selection was conducted based on validation macro F1. Using the published labeled test set, MARBERTv2 achieves the strongest independent result with 94.07% macro F1, followed by CAMeLBERT at 81.96% and AraBERTv2 at 79.77%. The results demonstrate the importance of Arabic pretraining and dialect-oriented representations and establish MARBERTv2 as the strongest standalone backbone in the current system.
Reliable Arabic hallucination detection requires not only strong ranking performance but also robustness to distribution shift, calibrated probabilities, and useful uncertainty for selective prediction. We present the NAMAA Community’s submission to HalluScoring 2026 Task ~1.1 and a set of post-submission diagnostic experiments. The scored, template-compliant system is a CAMeLBERT cross-encoder over the question and generated answer; it achieves 0.7596 ROC-AUC on the hidden test set, ranking third among seven teams. We compare this template-defined input path with reference-aware answer-pair models that use the gold answer. A CAMeLBERT input-framing ablation raises the unseen-question ROC-AUC from 0.8713 to 0.9272, while similar gains appear across ARBERT and MARBERTv2. Reference-free distillation recovers only a small portion of this advantage, whereas LoRA-InfoNCE substantially improves direct cosine discrimination. On a separately trained CAMeLBERT reliability replicate, temperature scaling reduces expected calibration error from 0.0993 to 0.0562, and MC-Dropout variance reduces retained-set error from 14.08% to 8.35% when the 30% most uncertain cases are deferred. These results motivate reporting distributional generalization, calibration, and selective risk alongside ROC-AUC. Code and checkpoints for the scored submission are publicly available. [<https://github.com/NAMAA-ORG/HalluScoring-Arabic-Hallucination-Detection>]