Annotating with Pros and Cons of Technologies in Computer Science Papers

Hono Shirai, Naoya Inoue, Jun Suzuki, Kentaro Inui


Abstract
This paper explores a task for extracting a technological expression and its pros/cons from computer science papers. We report ongoing efforts on an annotated corpus of pros/cons and an analysis of the nature of the automatic extraction task. Specifically, we show how to adapt the targeted sentiment analysis task for pros/cons extraction in computer science papers and conduct an annotation study. In order to identify the challenges of the automatic extraction task, we construct a strong baseline model and conduct an error analysis. The experiments show that pros/cons can be consistently annotated by several annotators, and that the task is challenging due to domain-specific knowledge. The annotated dataset is made publicly available for research purposes.
Anthology ID:
W19-2605
Volume:
Proceedings of the Workshop on Extracting Structured Knowledge from Scientific Publications
Month:
June
Year:
2019
Address:
Minneapolis, Minnesota
Editors:
Vivi Nastase, Benjamin Roth, Laura Dietz, Andrew McCallum
Venue:
NAACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
37–42
Language:
URL:
https://aclanthology.org/W19-2605
DOI:
10.18653/v1/W19-2605
Bibkey:
Cite (ACL):
Hono Shirai, Naoya Inoue, Jun Suzuki, and Kentaro Inui. 2019. Annotating with Pros and Cons of Technologies in Computer Science Papers. In Proceedings of the Workshop on Extracting Structured Knowledge from Scientific Publications, pages 37–42, Minneapolis, Minnesota. Association for Computational Linguistics.
Cite (Informal):
Annotating with Pros and Cons of Technologies in Computer Science Papers (Shirai et al., NAACL 2019)
Copy Citation:
PDF:
https://aclanthology.org/W19-2605.pdf
Code
 cl-tohoku/scientific-paper-pros-cons