@inproceedings{alansari-etal-2026-halluscoring,
title = "Halluscoring 2026: The First Shared Task on {LLM} Hallucination Detection Answer Verification",
author = "Alansari, Aisha and
Bouchekif, Abdessalam and
Hasanaath, Ahmed Abul and
Eddine Bekhouche, Salah and
Alkhorasani, Malak and
Zighem, Mohammed-En-Nadhir and
Ezzini, Saad and
Telli, Hichem and
Al-Khalifa, Hend and
Abdul-Mageed, Muhammad and
Hadid, Abdenour and
Luqman, Hamzah",
editor = "Abdul-Mageed, Muhammad and
Elsayed, Tamer and
Bouamor, Houda and
El-Beltagy, Samhaa and
Al-Khalifa, Hend and
Abdelali, Ahmed and
Elmadany, AbdelRahim and
El Mekki, Abdellah and
Darwish, Kareem and
Alkhereyf, Sakhar and
Luqman, Hamzah and
Khalifa, Salam and
Magdy, Samar and
Alyafeai, Zaid and
Hamed, Injy and
Zaghouani, Wajdi and
Zitouni, Imed",
booktitle = "Proceedings of The Fourth {A}rabic Natural Language Processing Conference: Shared Tasks",
month = oct,
year = "2026",
address = "Budapest, Hungary",
publisher = "Association for Computational Linguistics",
url = "https://aclanthology.org/2026.arabicnlp-sharedtasks.137/",
pages = "1106--1119",
ISBN = "979-8-89176-497-2",
abstract = "We present HalluScoring 2026, a shared task for evaluating hallucination detection and factual verification in Arabic question answering under challenging generalization settings. The shared task is organized into two main tasks, each comprising two subtasks, for a total of four subtasks. Task 1 evaluates binary hallucination detection, considering generalization to unseen questions (Subtask 1.1) and responses generated by unseen LLMs (Subtask 1.2). Task 2 extends the evaluation beyond detection by requiring the systems to additionally identify the correct factual answer from six related candidates, covering Islamic knowledge (Subtask 2.1) and general knowledge (Subtask 2.2). The shared task is based on two Arabic datasets: HalluScore and HalluTruthQA. A total of 13 teams participated in the shared task, 10 of which submitted system description papers. The results of Task 1 demonstrate that hallucination detection remains challenging under distribution shift, with the winning team achieving AUC-ROC test scores of 0.772 and 0.767 for Subtasks 1.1 and 1.2, respectively. For Task 2, the winning team achieved scores of 0.882 and 0.857 in the Islamic and general-knowledge subtasks, respectively, under assisted evaluation."
}<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="alansari-etal-2026-halluscoring">
<titleInfo>
<title>Halluscoring 2026: The First Shared Task on LLM Hallucination Detection Answer Verification</title>
</titleInfo>
<name type="personal">
<namePart type="given">Aisha</namePart>
<namePart type="family">Alansari</namePart>
<role>
<roleTerm authority="marcrelator" type="text">author</roleTerm>
</role>
</name>
<name type="personal">
<namePart type="given">Abdessalam</namePart>
<namePart type="family">Bouchekif</namePart>
<role>
<roleTerm authority="marcrelator" type="text">author</roleTerm>
</role>
</name>
<name type="personal">
<namePart type="given">Ahmed</namePart>
<namePart type="given">Abul</namePart>
<namePart type="family">Hasanaath</namePart>
<role>
<roleTerm authority="marcrelator" type="text">author</roleTerm>
</role>
</name>
<name type="personal">
<namePart type="given">Salah</namePart>
<namePart type="family">Eddine Bekhouche</namePart>
<role>
<roleTerm authority="marcrelator" type="text">author</roleTerm>
</role>
</name>
<name type="personal">
<namePart type="given">Malak</namePart>
<namePart type="family">Alkhorasani</namePart>
<role>
<roleTerm authority="marcrelator" type="text">author</roleTerm>
</role>
</name>
<name type="personal">
<namePart type="given">Mohammed-En-Nadhir</namePart>
<namePart type="family">Zighem</namePart>
<role>
<roleTerm authority="marcrelator" type="text">author</roleTerm>
</role>
</name>
<name type="personal">
<namePart type="given">Saad</namePart>
<namePart type="family">Ezzini</namePart>
<role>
<roleTerm authority="marcrelator" type="text">author</roleTerm>
</role>
</name>
<name type="personal">
<namePart type="given">Hichem</namePart>
<namePart type="family">Telli</namePart>
<role>
<roleTerm authority="marcrelator" type="text">author</roleTerm>
</role>
</name>
<name type="personal">
<namePart type="given">Hend</namePart>
<namePart type="family">Al-Khalifa</namePart>
<role>
<roleTerm authority="marcrelator" type="text">author</roleTerm>
</role>
</name>
<name type="personal">
<namePart type="given">Muhammad</namePart>
<namePart type="family">Abdul-Mageed</namePart>
<role>
<roleTerm authority="marcrelator" type="text">author</roleTerm>
</role>
</name>
<name type="personal">
<namePart type="given">Abdenour</namePart>
<namePart type="family">Hadid</namePart>
<role>
<roleTerm authority="marcrelator" type="text">author</roleTerm>
</role>
</name>
<name type="personal">
<namePart type="given">Hamzah</namePart>
<namePart type="family">Luqman</namePart>
<role>
<roleTerm authority="marcrelator" type="text">author</roleTerm>
</role>
</name>
<originInfo>
<dateIssued>2026-10</dateIssued>
</originInfo>
<typeOfResource>text</typeOfResource>
<relatedItem type="host">
<titleInfo>
<title>Proceedings of The Fourth Arabic Natural Language Processing Conference: Shared Tasks</title>
</titleInfo>
<name type="personal">
<namePart type="given">Muhammad</namePart>
<namePart type="family">Abdul-Mageed</namePart>
<role>
<roleTerm authority="marcrelator" type="text">editor</roleTerm>
</role>
</name>
<name type="personal">
<namePart type="given">Tamer</namePart>
<namePart type="family">Elsayed</namePart>
<role>
<roleTerm authority="marcrelator" type="text">editor</roleTerm>
</role>
</name>
<name type="personal">
<namePart type="given">Houda</namePart>
<namePart type="family">Bouamor</namePart>
<role>
<roleTerm authority="marcrelator" type="text">editor</roleTerm>
</role>
</name>
<name type="personal">
<namePart type="given">Samhaa</namePart>
<namePart type="family">El-Beltagy</namePart>
<role>
<roleTerm authority="marcrelator" type="text">editor</roleTerm>
</role>
</name>
<name type="personal">
<namePart type="given">Hend</namePart>
<namePart type="family">Al-Khalifa</namePart>
<role>
<roleTerm authority="marcrelator" type="text">editor</roleTerm>
</role>
</name>
<name type="personal">
<namePart type="given">Ahmed</namePart>
<namePart type="family">Abdelali</namePart>
<role>
<roleTerm authority="marcrelator" type="text">editor</roleTerm>
</role>
</name>
<name type="personal">
<namePart type="given">AbdelRahim</namePart>
<namePart type="family">Elmadany</namePart>
<role>
<roleTerm authority="marcrelator" type="text">editor</roleTerm>
</role>
</name>
<name type="personal">
<namePart type="given">Abdellah</namePart>
<namePart type="family">El Mekki</namePart>
<role>
<roleTerm authority="marcrelator" type="text">editor</roleTerm>
</role>
</name>
<name type="personal">
<namePart type="given">Kareem</namePart>
<namePart type="family">Darwish</namePart>
<role>
<roleTerm authority="marcrelator" type="text">editor</roleTerm>
</role>
</name>
<name type="personal">
<namePart type="given">Sakhar</namePart>
<namePart type="family">Alkhereyf</namePart>
<role>
<roleTerm authority="marcrelator" type="text">editor</roleTerm>
</role>
</name>
<name type="personal">
<namePart type="given">Hamzah</namePart>
<namePart type="family">Luqman</namePart>
<role>
<roleTerm authority="marcrelator" type="text">editor</roleTerm>
</role>
</name>
<name type="personal">
<namePart type="given">Salam</namePart>
<namePart type="family">Khalifa</namePart>
<role>
<roleTerm authority="marcrelator" type="text">editor</roleTerm>
</role>
</name>
<name type="personal">
<namePart type="given">Samar</namePart>
<namePart type="family">Magdy</namePart>
<role>
<roleTerm authority="marcrelator" type="text">editor</roleTerm>
</role>
</name>
<name type="personal">
<namePart type="given">Zaid</namePart>
<namePart type="family">Alyafeai</namePart>
<role>
<roleTerm authority="marcrelator" type="text">editor</roleTerm>
</role>
</name>
<name type="personal">
<namePart type="given">Injy</namePart>
<namePart type="family">Hamed</namePart>
<role>
<roleTerm authority="marcrelator" type="text">editor</roleTerm>
</role>
</name>
<name type="personal">
<namePart type="given">Wajdi</namePart>
<namePart type="family">Zaghouani</namePart>
<role>
<roleTerm authority="marcrelator" type="text">editor</roleTerm>
</role>
</name>
<name type="personal">
<namePart type="given">Imed</namePart>
<namePart type="family">Zitouni</namePart>
<role>
<roleTerm authority="marcrelator" type="text">editor</roleTerm>
</role>
</name>
<originInfo>
<publisher>Association for Computational Linguistics</publisher>
<place>
<placeTerm type="text">Budapest, Hungary</placeTerm>
</place>
</originInfo>
<genre authority="marcgt">conference publication</genre>
<identifier type="isbn">979-8-89176-497-2</identifier>
</relatedItem>
<abstract>We present HalluScoring 2026, a shared task for evaluating hallucination detection and factual verification in Arabic question answering under challenging generalization settings. The shared task is organized into two main tasks, each comprising two subtasks, for a total of four subtasks. Task 1 evaluates binary hallucination detection, considering generalization to unseen questions (Subtask 1.1) and responses generated by unseen LLMs (Subtask 1.2). Task 2 extends the evaluation beyond detection by requiring the systems to additionally identify the correct factual answer from six related candidates, covering Islamic knowledge (Subtask 2.1) and general knowledge (Subtask 2.2). The shared task is based on two Arabic datasets: HalluScore and HalluTruthQA. A total of 13 teams participated in the shared task, 10 of which submitted system description papers. The results of Task 1 demonstrate that hallucination detection remains challenging under distribution shift, with the winning team achieving AUC-ROC test scores of 0.772 and 0.767 for Subtasks 1.1 and 1.2, respectively. For Task 2, the winning team achieved scores of 0.882 and 0.857 in the Islamic and general-knowledge subtasks, respectively, under assisted evaluation.</abstract>
<identifier type="citekey">alansari-etal-2026-halluscoring</identifier>
<location>
<url>https://aclanthology.org/2026.arabicnlp-sharedtasks.137/</url>
</location>
<part>
<date>2026-10</date>
<extent unit="page">
<start>1106</start>
<end>1119</end>
</extent>
</part>
</mods>
</modsCollection>
%0 Conference Proceedings
%T Halluscoring 2026: The First Shared Task on LLM Hallucination Detection Answer Verification
%A Alansari, Aisha
%A Bouchekif, Abdessalam
%A Hasanaath, Ahmed Abul
%A Eddine Bekhouche, Salah
%A Alkhorasani, Malak
%A Zighem, Mohammed-En-Nadhir
%A Ezzini, Saad
%A Telli, Hichem
%A Al-Khalifa, Hend
%A Abdul-Mageed, Muhammad
%A Hadid, Abdenour
%A Luqman, Hamzah
%Y Abdul-Mageed, Muhammad
%Y Elsayed, Tamer
%Y Bouamor, Houda
%Y El-Beltagy, Samhaa
%Y Al-Khalifa, Hend
%Y Abdelali, Ahmed
%Y Elmadany, AbdelRahim
%Y El Mekki, Abdellah
%Y Darwish, Kareem
%Y Alkhereyf, Sakhar
%Y Luqman, Hamzah
%Y Khalifa, Salam
%Y Magdy, Samar
%Y Alyafeai, Zaid
%Y Hamed, Injy
%Y Zaghouani, Wajdi
%Y Zitouni, Imed
%S Proceedings of The Fourth Arabic Natural Language Processing Conference: Shared Tasks
%D 2026
%8 October
%I Association for Computational Linguistics
%C Budapest, Hungary
%@ 979-8-89176-497-2
%F alansari-etal-2026-halluscoring
%X We present HalluScoring 2026, a shared task for evaluating hallucination detection and factual verification in Arabic question answering under challenging generalization settings. The shared task is organized into two main tasks, each comprising two subtasks, for a total of four subtasks. Task 1 evaluates binary hallucination detection, considering generalization to unseen questions (Subtask 1.1) and responses generated by unseen LLMs (Subtask 1.2). Task 2 extends the evaluation beyond detection by requiring the systems to additionally identify the correct factual answer from six related candidates, covering Islamic knowledge (Subtask 2.1) and general knowledge (Subtask 2.2). The shared task is based on two Arabic datasets: HalluScore and HalluTruthQA. A total of 13 teams participated in the shared task, 10 of which submitted system description papers. The results of Task 1 demonstrate that hallucination detection remains challenging under distribution shift, with the winning team achieving AUC-ROC test scores of 0.772 and 0.767 for Subtasks 1.1 and 1.2, respectively. For Task 2, the winning team achieved scores of 0.882 and 0.857 in the Islamic and general-knowledge subtasks, respectively, under assisted evaluation.
%U https://aclanthology.org/2026.arabicnlp-sharedtasks.137/
%P 1106-1119
Markdown (Informal)
[Halluscoring 2026: The First Shared Task on LLM Hallucination Detection Answer Verification](https://aclanthology.org/2026.arabicnlp-sharedtasks.137/) (Alansari et al., ArabicNLP 2026)
ACL
- Aisha Alansari, Abdessalam Bouchekif, Ahmed Abul Hasanaath, Salah Eddine Bekhouche, Malak Alkhorasani, Mohammed-En-Nadhir Zighem, Saad Ezzini, Hichem Telli, Hend Al-Khalifa, Muhammad Abdul-Mageed, Abdenour Hadid, and Hamzah Luqman. 2026. Halluscoring 2026: The First Shared Task on LLM Hallucination Detection Answer Verification. In Proceedings of The Fourth Arabic Natural Language Processing Conference: Shared Tasks, pages 1106–1119, Budapest, Hungary. Association for Computational Linguistics.