Continual Few-shot Relation Learning via Embedding Space Regularization and Data Augmentation

Chengwei Qin, Shafiq Joty


Abstract
Existing continual relation learning (CRL) methods rely on plenty of labeled training data for learning a new task, which can be hard to acquire in real scenario as getting large and representative labeled data is often expensive and time-consuming. It is therefore necessary for the model to learn novel relational patterns with very few labeled data while avoiding catastrophic forgetting of previous task knowledge. In this paper, we formulate this challenging yet practical problem as continual few-shot relation learning (CFRL). Based on the finding that learning for new emerging few-shot tasks often results in feature distributions that are incompatible with previous tasks’ learned distributions, we propose a novel method based on embedding space regularization and data augmentation. Our method generalizes to new few-shot tasks and avoids catastrophic forgetting of previous tasks by enforcing extra constraints on the relational embeddings and by adding extra relevant data in a self-supervised manner. With extensive experiments we demonstrate that our method can significantly outperform previous state-of-the-art methods in CFRL task settings.
Anthology ID:
2022.acl-long.198
Volume:
Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
Month:
May
Year:
2022
Address:
Dublin, Ireland
Editors:
Smaranda Muresan, Preslav Nakov, Aline Villavicencio
Venue:
ACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
2776–2789
Language:
URL:
https://aclanthology.org/2022.acl-long.198
DOI:
10.18653/v1/2022.acl-long.198
Bibkey:
Cite (ACL):
Chengwei Qin and Shafiq Joty. 2022. Continual Few-shot Relation Learning via Embedding Space Regularization and Data Augmentation. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 2776–2789, Dublin, Ireland. Association for Computational Linguistics.
Cite (Informal):
Continual Few-shot Relation Learning via Embedding Space Regularization and Data Augmentation (Qin & Joty, ACL 2022)
Copy Citation:
PDF:
https://aclanthology.org/2022.acl-long.198.pdf
Software:
 2022.acl-long.198.software.zip
Video:
 https://aclanthology.org/2022.acl-long.198.mp4
Code
 qcwthu/continual_fewshot_relation_learning
Data
FewRel