Detecting Impact Relevant Sections in Scientific Research

Maria Becker, Kanyao Han, Haejin Lee, Antonina Werthmann, Rezvaneh Rezapour, Jana Diesner, Andreas Witt


Abstract
Impact assessment is an evolving area of research that aims at measuring and predicting the potential effects of projects or programs. Measuring the impact of scientific research is a vibrant subdomain, closely intertwined with impact assessment. A recurring obstacle pertains to the absence of an efficient framework which can facilitate the analysis of lengthy reports and text labeling. To address this issue, we propose a framework for automatically assessing the impact of scientific research projects by identifying pertinent sections in project reports that indicate the potential impacts. We leverage a mixed-method approach, combining manual annotations with supervised machine learning, to extract these passages from project reports. We experiment with different machine learning algorithms, including traditional statistical models as well as pre-trained transformer language models. Our experiments show that our proposed method achieves accuracy scores up to 0.81, and that our method is generalizable to scientific research from different domains and different languages.
Anthology ID:
2024.lrec-main.424
Volume:
Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024)
Month:
May
Year:
2024
Address:
Torino, Italia
Editors:
Nicoletta Calzolari, Min-Yen Kan, Veronique Hoste, Alessandro Lenci, Sakriani Sakti, Nianwen Xue
Venues:
LREC | COLING
SIG:
Publisher:
ELRA and ICCL
Note:
Pages:
4744–4749
Language:
URL:
https://aclanthology.org/2024.lrec-main.424
DOI:
Bibkey:
Cite (ACL):
Maria Becker, Kanyao Han, Haejin Lee, Antonina Werthmann, Rezvaneh Rezapour, Jana Diesner, and Andreas Witt. 2024. Detecting Impact Relevant Sections in Scientific Research. In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024), pages 4744–4749, Torino, Italia. ELRA and ICCL.
Cite (Informal):
Detecting Impact Relevant Sections in Scientific Research (Becker et al., LREC-COLING 2024)
Copy Citation:
PDF:
https://aclanthology.org/2024.lrec-main.424.pdf
Optional supplementary material:
 2024.lrec-main.424.OptionalSupplementaryMaterial.pdf