Scientific Statement Classification over arXiv.org

Deyan Ginev, Bruce R Miller


Abstract
We introduce a new classification task for scientific statements and release a large-scale dataset for supervised learning. Our resource is derived from a machine-readable representation of the arXiv.org collection of preprint articles. We explore fifty author-annotated categories and empirically motivate a task design of grouping 10.5 million annotated paragraphs into thirteen classes. We demonstrate that the task setup aligns with known success rates from the state of the art, peaking at a 0.91 F1-score via a BiLSTM encoder-decoder model. Additionally, we introduce a lexeme serialization for mathematical formulas, and observe that context-aware models could improve when also trained on the symbolic modality. Finally, we discuss the limitations of both data and task design, and outline potential directions towards increasingly complex models of scientific discourse, beyond isolated statements.
Anthology ID:
2020.lrec-1.153
Volume:
Proceedings of the Twelfth Language Resources and Evaluation Conference
Month:
May
Year:
2020
Address:
Marseille, France
Editors:
Nicoletta Calzolari, Frédéric Béchet, Philippe Blache, Khalid Choukri, Christopher Cieri, Thierry Declerck, Sara Goggi, Hitoshi Isahara, Bente Maegaard, Joseph Mariani, Hélène Mazo, Asuncion Moreno, Jan Odijk, Stelios Piperidis
Venue:
LREC
SIG:
Publisher:
European Language Resources Association
Note:
Pages:
1219–1226
Language:
English
URL:
https://aclanthology.org/2020.lrec-1.153
DOI:
Bibkey:
Cite (ACL):
Deyan Ginev and Bruce R Miller. 2020. Scientific Statement Classification over arXiv.org. In Proceedings of the Twelfth Language Resources and Evaluation Conference, pages 1219–1226, Marseille, France. European Language Resources Association.
Cite (Informal):
Scientific Statement Classification over arXiv.org (Ginev & Miller, LREC 2020)
Copy Citation:
PDF:
https://aclanthology.org/2020.lrec-1.153.pdf
Code
 dginev/arxiv-statement-classification +  additional community code
Data
Scientific statement classification dataset from arXMLiv 08.2018arXMLiv:08.2018