ArGuard Shared Task: Harmful Content Detection in Arabic Memes and LLM Prompts

Firoj Alam, Md. Rafiul Biswas, Mohamed Bayan Kmainasi, Ali Ezzat Shahroor, Hamdy Mubarak, George Mikros, Abul Hasnat, Wajdi Zaghouani


Abstract
Harmful content appears in diverse forms, ranging from multimodal memes targeting protected groups to textual prompts designed to elicit unsafe responses from large language models. However, existing Arabic resources typically study these settings separately and often rely on coarse-grained labels. We introduce **ArGuard**, a shared task on harmful content detection in Arabic memes and LLM prompts. ArGuard consists of two tracks: Track A focuses on multimodal hate detection in Arabic memes, while Track B addresses harmful prompt detection for Arabic LLM safety evaluation. Overall, 58 teams registered for the shared task, 35 participated in the final evaluation phase, and 27 submitted system-description papers. Participating systems explored a range of approaches, including fine-tuning encoder- and decoder-based models such as AraBERT, Jais, and Qwen3-VL. The best-performing systems achieved macro-F1 scores of 0.823 on Subtask A1, 0.419 on Subtask A2, 0.984 on Subtask B1, and 0.790 on Subtask B2. These results highlight fine-grained meme classification (Subtask A2) as the most challenging setting, partly due to sparse labels and train-test distribution shifts. To facilitate further research on Arabic harmful-content detection, we release the task datasets and associated resources through the shared-task website.
Anthology ID:
2026.arabicnlp-sharedtasks.39
Volume:
Proceedings of The Fourth Arabic Natural Language Processing Conference: Shared Tasks
Month:
October
Year:
2026
Address:
Budapest, Hungary
Editors:
Muhammad Abdul-Mageed, Tamer Elsayed, Houda Bouamor, Samhaa El-Beltagy, Hend Al-Khalifa, Ahmed Abdelali, AbdelRahim Elmadany, Abdellah El Mekki, Kareem Darwish, Sakhar Alkhereyf, Hamzah Luqman, Salam Khalifa, Samar Magdy, Zaid Alyafeai, Injy Hamed, Wajdi Zaghouani, Imed Zitouni
Venue:
ArabicNLP
SIG:
SIGARAB
Publisher:
Association for Computational Linguistics
Note:
Pages:
313–325
Language:
URL:
https://aclanthology.org/2026.arabicnlp-sharedtasks.39/
DOI:
Bibkey:
Cite (ACL):
Firoj Alam, Md. Rafiul Biswas, Mohamed Bayan Kmainasi, Ali Ezzat Shahroor, Hamdy Mubarak, George Mikros, Abul Hasnat, and Wajdi Zaghouani. 2026. ArGuard Shared Task: Harmful Content Detection in Arabic Memes and LLM Prompts. In Proceedings of The Fourth Arabic Natural Language Processing Conference: Shared Tasks, pages 313–325, Budapest, Hungary. Association for Computational Linguistics.
Cite (Informal):
ArGuard Shared Task: Harmful Content Detection in Arabic Memes and LLM Prompts (Alam et al., ArabicNLP 2026)
Copy Citation:
PDF:
https://aclanthology.org/2026.arabicnlp-sharedtasks.39.pdf