PromptDA: Label-guided Data Augmentation for Prompt-based Few Shot Learners

Canyu Chen, Kai Shu


Abstract
Recent advances in large pre-trained language models (PLMs) lead to impressive gains on natural language understanding (NLU) tasks with task-specific fine-tuning. However, directly fine-tuning PLMs heavily relies on sufficient labeled training instances, which are usually hard to obtain. Prompt-based tuning on PLMs has shown to be powerful for various downstream few-shot tasks. Existing works studying prompt-based tuning for few-shot NLU tasks mainly focus on deriving proper label words with a verbalizer or generating prompt templates to elicit semantics from PLMs. In addition, conventional data augmentation strategies such as synonym substitution are also widely adopted in low-resource scenarios. However, the improvements they bring to prompt-based few-shot learning have been demonstrated to be marginal. Thus, an important research question arises as follows: how to design effective data augmentation methods for prompt-based few-shot tuning? To this end, considering the label semantics are essential in prompt-based tuning, we propose a novel label-guided data augmentation framework PromptDA, which exploits the enriched label semantic information for data augmentation. Extensive experiment results on few-shot text classification tasks show that our proposed framework achieves superior performances by effectively leveraging label semantics and data augmentation for natural language understanding.
Anthology ID:
2023.eacl-main.41
Volume:
Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics
Month:
May
Year:
2023
Address:
Dubrovnik, Croatia
Editors:
Andreas Vlachos, Isabelle Augenstein
Venue:
EACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
562–574
Language:
URL:
https://aclanthology.org/2023.eacl-main.41
DOI:
10.18653/v1/2023.eacl-main.41
Bibkey:
Cite (ACL):
Canyu Chen and Kai Shu. 2023. PromptDA: Label-guided Data Augmentation for Prompt-based Few Shot Learners. In Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics, pages 562–574, Dubrovnik, Croatia. Association for Computational Linguistics.
Cite (Informal):
PromptDA: Label-guided Data Augmentation for Prompt-based Few Shot Learners (Chen & Shu, EACL 2023)
Copy Citation:
PDF:
https://aclanthology.org/2023.eacl-main.41.pdf
Video:
 https://aclanthology.org/2023.eacl-main.41.mp4