SciPrompt: Knowledge-augmented Prompting for Fine-grained Categorization of Scientific Topics

Zhiwen You, Kanyao Han, Haotian Zhu, Bertram Ludaescher, Jana Diesner


Abstract
Prompt-based fine-tuning has become an essential method for eliciting information encoded in pre-trained language models for a variety of tasks, including text classification. For multi-class classification tasks, prompt-based fine-tuning under low-resource scenarios has resulted in performance levels comparable to those of fully fine-tuning methods. Previous studies have used crafted prompt templates and verbalizers, mapping from the label terms space to the class space, to solve the classification problem as a masked language modeling task. However, cross-domain and fine-grained prompt-based fine-tuning with an automatically enriched verbalizer remains unexplored, mainly due to the difficulty and costs of manually selecting domain label terms for the verbalizer, which requires humans with domain expertise. To address this challenge, we introduce SciPrompt, a framework designed to automatically retrieve scientific topic-related terms for low-resource text classification tasks. To this end, we select semantically correlated and domain-specific label terms within the context of scientific literature for verbalizer augmentation. Furthermore, we propose a new verbalization strategy that uses correlation scores as additional weights to enhance the prediction performance of the language model during model tuning. Our method outperforms state-of-the-art, prompt-based fine-tuning methods on scientific text classification tasks under few and zero-shot settings, especially in classifying fine-grained and emerging scientific topics.
Anthology ID:
2024.emnlp-main.350
Volume:
Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing
Month:
November
Year:
2024
Address:
Miami, Florida, USA
Editors:
Yaser Al-Onaizan, Mohit Bansal, Yun-Nung Chen
Venue:
EMNLP
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
6087–6104
Language:
URL:
https://aclanthology.org/2024.emnlp-main.350
DOI:
Bibkey:
Cite (ACL):
Zhiwen You, Kanyao Han, Haotian Zhu, Bertram Ludaescher, and Jana Diesner. 2024. SciPrompt: Knowledge-augmented Prompting for Fine-grained Categorization of Scientific Topics. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 6087–6104, Miami, Florida, USA. Association for Computational Linguistics.
Cite (Informal):
SciPrompt: Knowledge-augmented Prompting for Fine-grained Categorization of Scientific Topics (You et al., EMNLP 2024)
Copy Citation:
PDF:
https://aclanthology.org/2024.emnlp-main.350.pdf
Data:
 2024.emnlp-main.350.data.zip