Satori Khalid

Author directory

2026

We present the L3IA Morocco team’s system for shared Task B, designed for large language models and focused on detecting harmful prompts in Arabic. In this context, we define two subtasks: B1 and B2, which refer, respectively, to the binary safe/unsafe classification and the classification of harmfulness into 7 categories. Our system combines three fine-tuned Arabic encoders: AraBERTv2, CAMeLBERTmix, and MARBERT, using soft voting that takes into account the predominant classes. It achieves a macro-F1 score of 0.946 for B1 and 0.902 for B2 during the development phase. In contrast, performance drops to 0.639 for B1 and 0.758 for B2 in the blind test. We demonstrate that this generalization divergence is due to a "unsafe” overprediction bias resulting from an unbalanced training corpus (80% unsafe) and not due to alignment of identifiers or dialects. Finally, we observed a negative result: in zeroshot and few-shot, AceGPT-7B, a generative Arabic LLM, performed poorly compared to fine-tuned encoders (0.38-0.42 on Darija).