QMUL-SDS at SCIVER: Step-by-Step Binary Classification for Scientific Claim Verification

Xia Zeng, Arkaitz Zubiaga


Abstract
Scientific claim verification is a unique challenge that is attracting increasing interest. The SCIVER shared task offers a benchmark scenario to test and compare claim verification approaches by participating teams and consists in three steps: relevant abstract selection, rationale selection and label prediction. In this paper, we present team QMUL-SDS’s participation in the shared task. We propose an approach that performs scientific claim verification by doing binary classifications step-by-step. We trained a BioBERT-large classifier to select abstracts based on pairwise relevance assessments for each <claim, title of the abstract> and continued to train it to select rationales out of each retrieved abstract based on <claim, sentence>. We then propose a two-step setting for label prediction, i.e. first predicting “NOT_ENOUGH_INFO” or “ENOUGH_INFO”, then label those marked as “ENOUGH_INFO” as either “SUPPORT” or “CONTRADICT”. Compared to the baseline system, we achieve substantial improvements on the dev set. As a result, our team is the No. 4 team on the leaderboard.
Anthology ID:
2021.sdp-1.15
Volume:
Proceedings of the Second Workshop on Scholarly Document Processing
Month:
June
Year:
2021
Address:
Online
Editors:
Iz Beltagy, Arman Cohan, Guy Feigenblat, Dayne Freitag, Tirthankar Ghosal, Keith Hall, Drahomira Herrmannova, Petr Knoth, Kyle Lo, Philipp Mayr, Robert M. Patton, Michal Shmueli-Scheuer, Anita de Waard, Kuansan Wang, Lucy Lu Wang
Venue:
sdp
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
116–123
Language:
URL:
https://aclanthology.org/2021.sdp-1.15
DOI:
10.18653/v1/2021.sdp-1.15
Bibkey:
Cite (ACL):
Xia Zeng and Arkaitz Zubiaga. 2021. QMUL-SDS at SCIVER: Step-by-Step Binary Classification for Scientific Claim Verification. In Proceedings of the Second Workshop on Scholarly Document Processing, pages 116–123, Online. Association for Computational Linguistics.
Cite (Informal):
QMUL-SDS at SCIVER: Step-by-Step Binary Classification for Scientific Claim Verification (Zeng & Zubiaga, sdp 2021)
Copy Citation:
PDF:
https://aclanthology.org/2021.sdp-1.15.pdf
Code
 XiaZeng0223/sciverbinary
Data
FEVERSciFact