A Multitask Active Learning Framework for Natural Language Understanding

Hua Zhu, Wu Ye, Sihan Luo, Xidong Zhang


Abstract
Natural language understanding (NLU) aims at identifying user intent and extracting semantic slots. This requires sufficient annotating data to get considerable performance in real-world situations. Active learning (AL) has been well-studied to decrease the needed amount of the annotating data and successfully applied to NLU. However, no research has been done on investigating how the relation information between intents and slots can improve the efficiency of AL algorithms. In this paper, we propose a multitask AL framework for NLU. Our framework enables pool-based AL algorithms to make use of the relation information between sub-tasks provided by a joint model, and we propose an efficient computation for the entropy of a joint model. Experimental results show our framework can achieve competitive performance with less training data than baseline methods on all datasets. We also demonstrate that when using the entropy as the query strategy, the model with complete relation information can perform better than those with partial information. Additionally, we demonstrate that the efficiency of these active learning algorithms in our framework is still effective when incorporate with the Bidirectional Encoder Representations from Transformers (BERT).
Anthology ID:
2020.coling-main.430
Volume:
Proceedings of the 28th International Conference on Computational Linguistics
Month:
December
Year:
2020
Address:
Barcelona, Spain (Online)
Editors:
Donia Scott, Nuria Bel, Chengqing Zong
Venue:
COLING
SIG:
Publisher:
International Committee on Computational Linguistics
Note:
Pages:
4900–4914
Language:
URL:
https://aclanthology.org/2020.coling-main.430
DOI:
10.18653/v1/2020.coling-main.430
Bibkey:
Cite (ACL):
Hua Zhu, Wu Ye, Sihan Luo, and Xidong Zhang. 2020. A Multitask Active Learning Framework for Natural Language Understanding. In Proceedings of the 28th International Conference on Computational Linguistics, pages 4900–4914, Barcelona, Spain (Online). International Committee on Computational Linguistics.
Cite (Informal):
A Multitask Active Learning Framework for Natural Language Understanding (Zhu et al., COLING 2020)
Copy Citation:
PDF:
https://aclanthology.org/2020.coling-main.430.pdf
Data
SNIPS