KG-FPQ: Evaluating Factuality Hallucination in LLMs with Knowledge Graph-based False Premise Questions

Yanxu Zhu, Jinlin Xiao, Yuhang Wang, Jitao Sang


Abstract
Recent studies have demonstrated that large language models (LLMs) are susceptible to being misled by false premise questions (FPQs), leading to errors in factual knowledge, known as factuality hallucination. Existing benchmarks that assess this vulnerability primarily rely on manual construction, resulting in limited size and lack of expandability. In this work, we introduce an automated, scalable pipeline to create FPQs based on knowledge graphs (KGs). The first step is to modify true triplets extracted from KGs to create false premises. Subsequently, utilizing the state-of-the-art capabilities of GPTs, we generate semantically rich FPQs. Based on the proposed method, we present a comprehensive benchmark, the Knowledge Graph-based False Premise Questions (KG-FPQ), which contains approximately 178k FPQs across three knowledge domains, at six levels of confusability, and in two task formats. Using KG-FPQ, we conduct extensive evaluations on several representative LLMs and provide valuable insights. The KG-FPQ dataset and code are available at https://github.com/yanxuzhu/KG-FPQ.
Anthology ID:
2025.coling-main.698
Volume:
Proceedings of the 31st International Conference on Computational Linguistics
Month:
January
Year:
2025
Address:
Abu Dhabi, UAE
Editors:
Owen Rambow, Leo Wanner, Marianna Apidianaki, Hend Al-Khalifa, Barbara Di Eugenio, Steven Schockaert
Venue:
COLING
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
10472–10490
Language:
URL:
https://aclanthology.org/2025.coling-main.698/
DOI:
Bibkey:
Cite (ACL):
Yanxu Zhu, Jinlin Xiao, Yuhang Wang, and Jitao Sang. 2025. KG-FPQ: Evaluating Factuality Hallucination in LLMs with Knowledge Graph-based False Premise Questions. In Proceedings of the 31st International Conference on Computational Linguistics, pages 10472–10490, Abu Dhabi, UAE. Association for Computational Linguistics.
Cite (Informal):
KG-FPQ: Evaluating Factuality Hallucination in LLMs with Knowledge Graph-based False Premise Questions (Zhu et al., COLING 2025)
Copy Citation:
PDF:
https://aclanthology.org/2025.coling-main.698.pdf