QQATeam at Qur’an QA 2022: Fine-Tunning Arabic QA Models for Qur’an QA Task

Basem Ahmed, Motaz Saad, Eshrag A. Refaee


Abstract
The problem of auto-extraction of reliable answers from a reference text like a constitution or holy book is a real challenge for the natural languages research community. Qurán is the holy book of Islam and the primary source of legislation for millions of Muslims around the world, which can trigger the curiosity of non-Muslims to find answers about various topics from the Qurán. Previous work on Question Answering (Q&A) from Qurán is scarce and lacks the benchmark of previously developed systems on a testbed to allow meaningful comparison and identify developments and challenges. This work presents an empirical investigation of our participation in the Qurán QA shared task (2022) that utilizes a benchmark dataset of 1,093 tuples of question-Qurán passage pairs. The dataset comprises Qurán verses, questions and several ranked possible answers. This paper describes the approach we follow with our participation in the shared task and summarises our main findings. Our system attained the best score at 0.63 pRR and 0.59 F1 on the development set and 0.56 pRR and 0.51 F1 on the test set. The best results of the Exact Match (EM) score at 0.34 indicate the difficulty of the task and the need for more future work to tackle this challenging task.
Anthology ID:
2022.osact-1.16
Volume:
Proceedinsg of the 5th Workshop on Open-Source Arabic Corpora and Processing Tools with Shared Tasks on Qur'an QA and Fine-Grained Hate Speech Detection
Month:
June
Year:
2022
Address:
Marseille, France
Editors:
Hend Al-Khalifa, Tamer Elsayed, Hamdy Mubarak, Abdulmohsen Al-Thubaity, Walid Magdy, Kareem Darwish
Venue:
OSACT
SIG:
Publisher:
European Language Resources Association
Note:
Pages:
130–135
Language:
URL:
https://aclanthology.org/2022.osact-1.16
DOI:
Bibkey:
Cite (ACL):
Basem Ahmed, Motaz Saad, and Eshrag A. Refaee. 2022. QQATeam at Qur’an QA 2022: Fine-Tunning Arabic QA Models for Qur’an QA Task. In Proceedinsg of the 5th Workshop on Open-Source Arabic Corpora and Processing Tools with Shared Tasks on Qur'an QA and Fine-Grained Hate Speech Detection, pages 130–135, Marseille, France. European Language Resources Association.
Cite (Informal):
QQATeam at Qur’an QA 2022: Fine-Tunning Arabic QA Models for Qur’an QA Task (Ahmed et al., OSACT 2022)
Copy Citation:
PDF:
https://aclanthology.org/2022.osact-1.16.pdf