Dual Context-Guided Continuous Prompt Tuning for Few-Shot Learning

Jie Zhou, Le Tian, Houjin Yu, Zhou Xiao, Hui Su, Jie Zhou


Abstract
Prompt-based paradigm has shown its competitive performance in many NLP tasks. However, its success heavily depends on prompt design, and the effectiveness varies upon the model and training data. In this paper, we propose a novel dual context-guided continuous prompt (DCCP) tuning method. To explore the rich contextual information in language structure and close the gap between discrete prompt tuning and continuous prompt tuning, DCCP introduces two auxiliary training objectives and constructs input in a pair-wise fashion. Experimental results demonstrate that our method is applicable to many NLP tasks, and can often outperform existing prompt tuning methods by a large margin in the few-shot setting.
Anthology ID:
2022.findings-acl.8
Volume:
Findings of the Association for Computational Linguistics: ACL 2022
Month:
May
Year:
2022
Address:
Dublin, Ireland
Editors:
Smaranda Muresan, Preslav Nakov, Aline Villavicencio
Venue:
Findings
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
79–84
Language:
URL:
https://aclanthology.org/2022.findings-acl.8
DOI:
10.18653/v1/2022.findings-acl.8
Bibkey:
Cite (ACL):
Jie Zhou, Le Tian, Houjin Yu, Zhou Xiao, Hui Su, and Jie Zhou. 2022. Dual Context-Guided Continuous Prompt Tuning for Few-Shot Learning. In Findings of the Association for Computational Linguistics: ACL 2022, pages 79–84, Dublin, Ireland. Association for Computational Linguistics.
Cite (Informal):
Dual Context-Guided Continuous Prompt Tuning for Few-Shot Learning (Zhou et al., Findings 2022)
Copy Citation:
PDF:
https://aclanthology.org/2022.findings-acl.8.pdf