PubHealthTab: A Public Health Table-based Dataset for Evidence-based Fact Checking

Mubashara Akhtar, Oana Cocarascu, Elena Simperl


Abstract
Inspired by human fact checkers, who use different types of evidence (e.g. tables, images, audio) in addition to text, several datasets with tabular evidence data have been released in recent years. Whilst the datasets encourage research on table fact-checking, they rely on information from restricted data sources, such as Wikipedia for creating claims and extracting evidence data, making the fact-checking process different from the real-world process used by fact checkers. In this paper, we introduce PubHealthTab, a table fact-checking dataset based on real world public health claims and noisy evidence tables from sources similar to those used by real fact checkers. We outline our approach for collecting evidence data from various websites and present an in-depth analysis of our dataset. Finally, we evaluate state-of-the-art table representation and pre-trained models fine-tuned on our dataset, achieving an overall F1 score of 0.73.
Anthology ID:
2022.findings-naacl.1
Volume:
Findings of the Association for Computational Linguistics: NAACL 2022
Month:
July
Year:
2022
Address:
Seattle, United States
Editors:
Marine Carpuat, Marie-Catherine de Marneffe, Ivan Vladimir Meza Ruiz
Venue:
Findings
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
1–16
Language:
URL:
https://aclanthology.org/2022.findings-naacl.1
DOI:
10.18653/v1/2022.findings-naacl.1
Bibkey:
Cite (ACL):
Mubashara Akhtar, Oana Cocarascu, and Elena Simperl. 2022. PubHealthTab: A Public Health Table-based Dataset for Evidence-based Fact Checking. In Findings of the Association for Computational Linguistics: NAACL 2022, pages 1–16, Seattle, United States. Association for Computational Linguistics.
Cite (Informal):
PubHealthTab: A Public Health Table-based Dataset for Evidence-based Fact Checking (Akhtar et al., Findings 2022)
Copy Citation:
PDF:
https://aclanthology.org/2022.findings-naacl.1.pdf
Software:
 2022.findings-naacl.1.software.zip
Video:
 https://aclanthology.org/2022.findings-naacl.1.mp4
Code
 mubasharaak/pubhealthtab
Data
BLUEFEVERFEVEROUSInfoTabSMultiNLIPUBHEALTHSNLI