XInfoTabS: Evaluating Multilingual Tabular Natural Language Inference

Bhavnick Minhas, Anant Shankhdhar, Vivek Gupta, Divyanshu Aggarwal, Shuo Zhang


Abstract
The ability to reason about tabular or semi-structured knowledge is a fundamental problem for today’s Natural Language Processing (NLP) systems. While significant progress has been achieved in the direction of tabular reasoning, these advances are limited to English due to the absence of multilingual benchmark datasets for semi-structured data. In this paper, we use machine translation methods to construct a multilingual tabular NLI dataset, namely XINFOTABS, which expands the English tabular NLI dataset of INFOTABS to ten diverse languages. We also present several baselines for multilingual tabular reasoning, e.g., machine translation-based methods and cross-lingual. We discover that the XINFOTABS evaluation suite is both practical and challenging. As a result, this dataset will contribute to increased linguistic inclusion in tabular reasoning research and applications.
Anthology ID:
2022.fever-1.7
Volume:
Proceedings of the Fifth Fact Extraction and VERification Workshop (FEVER)
Month:
May
Year:
2022
Address:
Dublin, Ireland
Editors:
Rami Aly, Christos Christodoulopoulos, Oana Cocarascu, Zhijiang Guo, Arpit Mittal, Michael Schlichtkrull, James Thorne, Andreas Vlachos
Venue:
FEVER
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
59–77
Language:
URL:
https://aclanthology.org/2022.fever-1.7
DOI:
10.18653/v1/2022.fever-1.7
Bibkey:
Cite (ACL):
Bhavnick Minhas, Anant Shankhdhar, Vivek Gupta, Divyanshu Aggarwal, and Shuo Zhang. 2022. XInfoTabS: Evaluating Multilingual Tabular Natural Language Inference. In Proceedings of the Fifth Fact Extraction and VERification Workshop (FEVER), pages 59–77, Dublin, Ireland. Association for Computational Linguistics.
Cite (Informal):
XInfoTabS: Evaluating Multilingual Tabular Natural Language Inference (Minhas et al., FEVER 2022)
Copy Citation:
PDF:
https://aclanthology.org/2022.fever-1.7.pdf
Video:
 https://aclanthology.org/2022.fever-1.7.mp4
Data
TabFact