This paper presents our participation in IslamicEval 2026 Subtask 4, citation relevance classification for Qur’an and Hadith spans. We propose two independent systems for this task. The first is a generative large language model prompted to produce a structured, step by step relevance judgment. The second is a domain pretrained sentence encoder fine tuned with triplet loss on citation relevance data. Our evaluation compares two generative backends and three encoders pretrained on different Arabic varieties, using macro averaged F1 as the metric. Results show on development set that chain of thought judgment with ALLaM-7B consistently outperforms our embedding similarity ensemble, reaching 0.774 macro F1 against 0.357 for the ensemble, and outperforms an equivalent Fanar-S-1-7B system on seven of eight prompting strategies. Additionally, AraBERT-v02, pretrained on Modern Standard Arabic, is the consistently strongest encoder, outperforming Classical Arabic pretrained encoders in this citation relevance task. Our best submitted system, ALLaM with few shot prompting without chain of thought, scores 0.748 macro F1 on the blind test set, narrowly ahead of the same backend with chain of thought added, at 0.744.

Question answering is a specialized area in the field of NLP that aims to extract the answer to a user question from a given text. Most studies in this area focus on the English language, while other languages, such as Arabic, are still in their early stage. Recently, research tend to develop question answering systems for Arabic Islamic texts, which may impose challenges due to Classical Arabic. In this paper, we use Simple Transformers Question Answering model with three Arabic pre-trained language models (AraBERT, CAMeL-BERT, ArabicBERT) for Qur’an Question Answering task using Qur’anic Reading Comprehension Dataset. The model is set to return five answers ranking from the best to worst based on their probability scores according to the task details. Our experiments with development set shows that AraBERT V0.2 model outperformed the other Arabic pre-trained models. Therefore, AraBERT V0.2 was chosen for the the test set and it performed fair results with 0.45 pRR score, 0.16 EM score and 0.42 F1 score.