TabVer: Tabular Fact Verification with Natural Logic

Rami Aly, Andreas Vlachos


Abstract
Fact verification on tabular evidence incentivizes the use of symbolic reasoning models where a logical form is constructed (e.g., a LISP-style program), providing greater verifiability than fully neural approaches. However, these logical forms typically rely on well-formed tables, restricting their use in many scenarios. An emerging symbolic reasoning paradigm for textual evidence focuses on natural logic inference, which constructs proofs by modeling set-theoretic relations between a claim and its evidence in natural language. This approach provides flexibility and transparency but is less compatible with tabular evidence since the relations do not extend to arithmetic functions. We propose a set-theoretic interpretation of numerals and arithmetic functions in the context of natural logic, enabling the integration of arithmetic expressions in deterministic proofs. We leverage large language models to generate arithmetic expressions by generating questions about salient parts of a claim which are answered by executing appropriate functions on tables. In a few-shot setting on FEVEROUS, we achieve an accuracy of 71.4, outperforming both fully neural and symbolic reasoning models by 3.4 points. When evaluated on TabFact without any further training, our method remains competitive with an accuracy lead of 0.5 points.
Anthology ID:
2024.tacl-1.89
Volume:
Transactions of the Association for Computational Linguistics, Volume 12
Month:
Year:
2024
Address:
Cambridge, MA
Venue:
TACL
SIG:
Publisher:
MIT Press
Note:
Pages:
1648–1671
Language:
URL:
https://aclanthology.org/2024.tacl-1.89/
DOI:
10.1162/tacl_a_00722
Bibkey:
Cite (ACL):
Rami Aly and Andreas Vlachos. 2024. TabVer: Tabular Fact Verification with Natural Logic. Transactions of the Association for Computational Linguistics, 12:1648–1671.
Cite (Informal):
TabVer: Tabular Fact Verification with Natural Logic (Aly & Vlachos, TACL 2024)
Copy Citation:
PDF:
https://aclanthology.org/2024.tacl-1.89.pdf