Learning to Classify Intents and Slot Labels Given a Handful of Examples

Jason Krone, Yi Zhang, Mona Diab


Abstract
Intent classification (IC) and slot filling (SF) are core components in most goal-oriented dialogue systems. Current IC/SF models perform poorly when the number of training examples per class is small. We propose a new few-shot learning task, few-shot IC/SF, to study and improve the performance of IC and SF models on classes not seen at training time in ultra low resource scenarios. We establish a few-shot IC/SF benchmark by defining few-shot splits for three public IC/SF datasets, ATIS, TOP, and Snips. We show that two popular few-shot learning algorithms, model agnostic meta learning (MAML) and prototypical networks, outperform a fine-tuning baseline on this benchmark. Prototypical networks achieves significant gains in IC performance on the ATIS and TOP datasets, while both prototypical networks and MAML outperform the baseline with respect to SF on all three datasets. In addition, we demonstrate that joint training as well as the use of pre-trained language models, ELMo and BERT in our case, are complementary to these few-shot learning methods and yield further gains.
Anthology ID:
2020.nlp4convai-1.12
Volume:
Proceedings of the 2nd Workshop on Natural Language Processing for Conversational AI
Month:
July
Year:
2020
Address:
Online
Venues:
ACL | NLP4ConvAI | WS
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
96–108
Language:
URL:
https://aclanthology.org/2020.nlp4convai-1.12
DOI:
10.18653/v1/2020.nlp4convai-1.12
Bibkey:
Copy Citation:
PDF:
https://aclanthology.org/2020.nlp4convai-1.12.pdf
Video:
 http://slideslive.com/38929638