This paper presents a comprehensive overview of the QIAS 2026 shared task, organized as part of the OSACT7 Workshop and co-located with LREC 2026. The shared task was designed to evaluate the ability of large language models to perform complex reasoning in the religious and legal domain of Islamic inheritance. Unlike conventional question-answering benchmarks, QIAS 2026 focuses on end-to-end reasoning from natural language cases, requiring systems to perform the full inheritance calculation process, from identifying the eligible heirs to assigning the correct share to each beneficiary. To support this evaluation, the task was based on the MAWARITH benchmark, a dataset of 12,500 Arabic inheritance cases annotated with intermediate reasoning steps and final answers. System submissions were evaluated using MIR-E, a multi-step metric that measures performance across the main stages of inheritance reasoning. A total of 16 teams participated in the shared task, investigating a range of approaches, including prompting-based methods, retrieval-augmented generation, and fine-tuning strategies. The results show that Islamic inheritance remains a highly challenging benchmark for current language models, especially in stages that require precise legal interpretation and structured numerical reasoning. This overview summarizes the task design, dataset, evaluation framework, participating systems, and main results.

Retrieval-Augmented Generation is used for Islamic question answering, but most systems are evaluated end-to-end, making retrieval failures difficult to isolate from generation failures. We study answer-bearing retrieval for Arabic fiqh, where a passage is relevant only if it states the ruling required by the question. We build a retrieval test collection for Arabic fiqh and use it to evaluate dense, lexical, hybrid, fine-tuned, and madhhab-aware retrieval strategies. The best retriever achieves 0.524 MRR@5, while fine-tuning improves performance to 0.553. Hybrid retrieval provides limited gains for strong models, whereas madhhab-aware filtering more than doubles MRR@5 on school-specific questions. We further present an error analysis showing that the main challenge is distinguishing answer-bearing passages from topically similar passages that do not contain the requested ruling. The best-performing model and fine-tuning dataset are publicly available at https://huggingface.co/collections/somayaeltanbouly/badr-arabic-fiqh-retrieval

Large language models (LLMs) can generate fluent Arabic answers, yet factual errors remain difficult to detect, localize, explain, and verify. Existing hallucination benchmarks often provide only response-level labels, offering limited support for identifying the exact erroneous content, explaining why it is incorrect, or recovering the correct factual answer. We introduce HalluTruthQA, a fine-grained benchmark for hallucination evaluation in Arabic question answering. HalluTruthQA contains 2,400 expert-curated examples across four knowledge-intensive domains: Islamic knowledge, history, science, and geography. Each example includes an Arabic question, a model-generated answer, a verified reference answer, a binary hallucination label, and six candidate answers for factual verification. For hallucinated answers, the benchmark additionally provides character-level erroneous spans, human-written explanations, and macro- and micro-level hallucination types. We evaluate four open-source LLMs—ALLaM-7B, Falcon-H1R-7B, Qwen3-32B, and SILMA—in a zero-shot setting across four tasks: hallucination detection, span-level localization, factual verification, and explanation evaluation. The results show that these tasks measure complementary capabilities, with no single model achieving the best performance across all of them. The best observed scores are 0.880 Macro-F1 for hallucination detection, 0.516 F1-Sp for span localization, 0.852 LO-Score for factual verification, and 0.644 for explanation evaluation. Our findings highlight the importance of moving beyond binary hallucination detection toward fine-grained evaluation that also localizes, verifies, categorizes, and explains factual errors. Code, data, prompts, and evaluation scripts are available at https://gitlab.com/nlpresearcher/HalluTruthQA