Volta at SemEval-2021 Task 9: Statement Verification and Evidence Finding with Tables using TAPAS and Transfer Learning

Devansh Gautam, Kshitij Gupta, Manish Shrivastava


Abstract
Tables are widely used in various kinds of documents to present information concisely. Understanding tables is a challenging problem that requires an understanding of language and table structure, along with numerical and logical reasoning. In this paper, we present our systems to solve Task 9 of SemEval-2021: Statement Verification and Evidence Finding with Tables (SEM-TAB-FACTS). The task consists of two subtasks: (A) Given a table and a statement, predicting whether the table supports the statement and (B) Predicting which cells in the table provide evidence for/against the statement. We fine-tune TAPAS (a model which extends BERT’s architecture to capture tabular structure) for both the subtasks as it has shown state-of-the-art performance in various table understanding tasks. In subtask A, we evaluate how transfer learning and standardizing tables to have a single header row improves TAPAS’ performance. In subtask B, we evaluate how different fine-tuning strategies can improve TAPAS’ performance. Our systems achieve an F1 score of 67.34 in subtask A three-way classification, 72.89 in subtask A two-way classification, and 62.95 in subtask B.
Anthology ID:
2021.semeval-1.180
Volume:
Proceedings of the 15th International Workshop on Semantic Evaluation (SemEval-2021)
Month:
August
Year:
2021
Address:
Online
Editors:
Alexis Palmer, Nathan Schneider, Natalie Schluter, Guy Emerson, Aurelie Herbelot, Xiaodan Zhu
Venue:
SemEval
SIG:
SIGLEX
Publisher:
Association for Computational Linguistics
Note:
Pages:
1262–1270
Language:
URL:
https://aclanthology.org/2021.semeval-1.180
DOI:
10.18653/v1/2021.semeval-1.180
Bibkey:
Cite (ACL):
Devansh Gautam, Kshitij Gupta, and Manish Shrivastava. 2021. Volta at SemEval-2021 Task 9: Statement Verification and Evidence Finding with Tables using TAPAS and Transfer Learning. In Proceedings of the 15th International Workshop on Semantic Evaluation (SemEval-2021), pages 1262–1270, Online. Association for Computational Linguistics.
Cite (Informal):
Volta at SemEval-2021 Task 9: Statement Verification and Evidence Finding with Tables using TAPAS and Transfer Learning (Gautam et al., SemEval 2021)
Copy Citation:
PDF:
https://aclanthology.org/2021.semeval-1.180.pdf
Code
 devanshg27/sem-tab-fact
Data
SQATabFactWikiSQL