Neighbor Contextual Information Learners for Joint Intent and Slot Prediction

Bharatram Natarajan, Gaurav Mathur, Sameer Jain


Abstract
Intent Identification and Slot Identification aretwo important task for Natural Language Understanding(NLU). Exploration in this areahave gained significance using networks likeRNN, LSTM and GRU. However, modelscontaining the above modules are sequentialin nature, which consumes lot of resourceslike memory to train the model in cloud itself. With the advent of many voice assistantsdelivering offline solutions for manyapplications, there is a need for finding replacementfor such sequential networks. Explorationin self-attention, CNN modules hasgained pace in the recent times. Here we exploreCNN based models like Trellis and modifiedthe architecture to make it bi-directionalwith fusion techniques. In addition, we proposeCNN with Self Attention network calledNeighbor Contextual Information Projector usingMulti Head Attention (NCIPMA) architecture. These architectures beat state of the art inopen source datasets like ATIS, SNIPS.
Anthology ID:
2020.icon-workshop.1
Volume:
Proceedings of the Workshop on Joint NLP Modelling for Conversational AI @ ICON 2020
Month:
December
Year:
2020
Address:
Patna, India
Editors:
Praveen Kumar G S, Siddhartha Mukherjee, Ranjan Samal
Venue:
ICON
SIG:
Publisher:
NLP Association of India (NLPAI)
Note:
Pages:
1–9
Language:
URL:
https://aclanthology.org/2020.icon-workshop.1
DOI:
Bibkey:
Cite (ACL):
Bharatram Natarajan, Gaurav Mathur, and Sameer Jain. 2020. Neighbor Contextual Information Learners for Joint Intent and Slot Prediction. In Proceedings of the Workshop on Joint NLP Modelling for Conversational AI @ ICON 2020, pages 1–9, Patna, India. NLP Association of India (NLPAI).
Cite (Informal):
Neighbor Contextual Information Learners for Joint Intent and Slot Prediction (Natarajan et al., ICON 2020)
Copy Citation:
PDF:
https://aclanthology.org/2020.icon-workshop.1.pdf