Overview and Insights from the SCIVER shared task on Scientific Claim Verification

David Wadden, Kyle Lo


Abstract
We present an overview of the SCIVER shared task, presented at the 2nd Scholarly Document Processing (SDP) workshop at NAACL 2021. In this shared task, systems were provided a scientific claim and a corpus of research abstracts, and asked to identify which articles Support or Refute the claim as well as provide evidentiary sentences justifying those labels. 11 teams made a total of 14 submissions to the shared task leaderboard, leading to an improvement of more than +23 F1 on the primary task evaluation metric. In addition to surveying the participating systems, we provide several insights into modeling approaches to support continued progress and future research on the important and challenging task of scientific claim verification.
Anthology ID:
2021.sdp-1.16
Volume:
Proceedings of the Second Workshop on Scholarly Document Processing
Month:
June
Year:
2021
Address:
Online
Editors:
Iz Beltagy, Arman Cohan, Guy Feigenblat, Dayne Freitag, Tirthankar Ghosal, Keith Hall, Drahomira Herrmannova, Petr Knoth, Kyle Lo, Philipp Mayr, Robert M. Patton, Michal Shmueli-Scheuer, Anita de Waard, Kuansan Wang, Lucy Lu Wang
Venue:
sdp
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
124–129
Language:
URL:
https://aclanthology.org/2021.sdp-1.16
DOI:
Bibkey:
Cite (ACL):
David Wadden and Kyle Lo. 2021. Overview and Insights from the SCIVER shared task on Scientific Claim Verification. In Proceedings of the Second Workshop on Scholarly Document Processing, pages 124–129, Online. Association for Computational Linguistics.
Cite (Informal):
Overview and Insights from the SCIVER shared task on Scientific Claim Verification (Wadden & Lo, sdp 2021)
Copy Citation:
PDF:
https://aclanthology.org/2021.sdp-1.16.pdf
Data
FEVERSciFact