A Unified LLM-KG Framework to Assist Fact-Checking in Public Deliberation

Nikolaos Giarelis, Charalampos Mastrokostas, Nikos Karacapilidis


Abstract
Fact-checking plays a crucial role in public deliberation by promoting transparency, accuracy, credibility, and accountability. Aiming to augment the efficiency and adoption of current public deliberation platforms, which mostly rely on the abilities of participants to meaningfully process and interpret the associated content, this paper explores the combination of deep learning and symbolic reasoning. Specifically, it proposes a framework that unifies the capabilities of Large Language Models (LLMs) and Knowledge Graphs (KGs), and reports on an experimental evaluation. This evaluation is conducted through a questionnaire asking users to assess a baseline LLM against the proposed framework, using a series of fact-checking metrics, namely readability, coverage, non-redundancy, and quality. The experimentation results are promising and confirm the potential of combining the capabilities of these two technologies in the context of public deliberation and digital democracy.
Anthology ID:
2024.delite-1.2
Volume:
Proceedings of the First Workshop on Language-driven Deliberation Technology (DELITE) @ LREC-COLING 2024
Month:
May
Year:
2024
Address:
Torino, Italia
Editors:
Annette Hautli-Janisz, Gabriella Lapesa, Lucas Anastasiou, Valentin Gold, Anna De Liddo, Chris Reed
Venue:
DELITE
SIG:
Publisher:
ELRA and ICCL
Note:
Pages:
13–19
Language:
URL:
https://aclanthology.org/2024.delite-1.2
DOI:
Bibkey:
Cite (ACL):
Nikolaos Giarelis, Charalampos Mastrokostas, and Nikos Karacapilidis. 2024. A Unified LLM-KG Framework to Assist Fact-Checking in Public Deliberation. In Proceedings of the First Workshop on Language-driven Deliberation Technology (DELITE) @ LREC-COLING 2024, pages 13–19, Torino, Italia. ELRA and ICCL.
Cite (Informal):
A Unified LLM-KG Framework to Assist Fact-Checking in Public Deliberation (Giarelis et al., DELITE 2024)
Copy Citation:
PDF:
https://aclanthology.org/2024.delite-1.2.pdf