Large language models (LLMs) can generate fluent Arabic answers, yet factual errors remain difficult to detect, localize, explain, and verify. Existing hallucination benchmarks often provide only response-level labels, offering limited support for identifying the exact erroneous content, explaining why it is incorrect, or recovering the correct factual answer. We introduce HalluTruthQA, a fine-grained benchmark for hallucination evaluation in Arabic question answering. HalluTruthQA contains 2,400 expert-curated examples across four knowledge-intensive domains: Islamic knowledge, history, science, and geography. Each example includes an Arabic question, a model-generated answer, a verified reference answer, a binary hallucination label, and six candidate answers for factual verification. For hallucinated answers, the benchmark additionally provides character-level erroneous spans, human-written explanations, and macro- and micro-level hallucination types. We evaluate four open-source LLMs—ALLaM-7B, Falcon-H1R-7B, Qwen3-32B, and SILMA—in a zero-shot setting across four tasks: hallucination detection, span-level localization, factual verification, and explanation evaluation. The results show that these tasks measure complementary capabilities, with no single model achieving the best performance across all of them. The best observed scores are 0.880 Macro-F1 for hallucination detection, 0.516 F1-Sp for span localization, 0.852 LO-Score for factual verification, and 0.644 for explanation evaluation. Our findings highlight the importance of moving beyond binary hallucination detection toward fine-grained evaluation that also localizes, verifies, categorizes, and explains factual errors. Code, data, prompts, and evaluation scripts are available at https://gitlab.com/nlpresearcher/HalluTruthQA

This paper provides a comprehensive overview of the QIAS 2025 shared task, organized as part of the ArabicNLP 2025 conference and co-located with EMNLP 2025. The task was designed for the evaluation of large language models in the complex domains of religious and legal reasoning. It comprises two subtasks: (1) Islamic Inheritance Reasoning, requiring models to compute inheritance shares according to Islamic jurisprudence, and (2) Islamic Knowledge Assessment, which covers a range of traditional Islamic disciplines. Both subtasks were structured as multiple-choice question answering challenges, with questions stratified by varying difficulty levels. The shared task attracted significant interest, with 44 teams participating in the development phase, from which 18 teams advanced to the final test phase. Of these, 6 teams submitted entries for both subtasks, 8 for Task 1 only, and two for Task 3 only. Ultimately, 16 teams submitted system description papers. Herein, we detail the task’s motivation, dataset construction, evaluation protocol, and present a summary of the participating systems and their results.