Towards Explainable Dialogue System: Explaining Intent Classification using Saliency Techniques

Ratnesh Joshi, Arindam Chatterjee, Asif Ekbal


Abstract
Deep learning based methods have shown tremendous success in several Natural Language Processing (NLP) tasks. The recent trends in the usage of Deep Learning based models for natural language tasks have definitely produced incredible performance for several application areas. However, one major problem that most of these models face is the lack of transparency, i.e. the actual decision process of the underlying model is not explainable. In this paper, at first we solve a very fundamental problem of Natural Language Understanding (NLU), i.e. intent detection using a Bi-directional Long Short Term Memory (BiLSTM). In order to determine the defining features that lead to a specific intent class, we use the Layerwise Relevance Propagation (LRP) algorithm to find the defining feature(s). In the process, we conclude that saliency method of eLRP (epsilon Layerwise Relevance Propagation) is a prominent process for highlighting the important features of the input responsible for the current classification which results in significant insights to the inner workings, such as the reasons for misclassification by the black box model.
Anthology ID:
2021.icon-main.16
Volume:
Proceedings of the 18th International Conference on Natural Language Processing (ICON)
Month:
December
Year:
2021
Address:
National Institute of Technology Silchar, Silchar, India
Editors:
Sivaji Bandyopadhyay, Sobha Lalitha Devi, Pushpak Bhattacharyya
Venue:
ICON
SIG:
Publisher:
NLP Association of India (NLPAI)
Note:
Pages:
120–127
Language:
URL:
https://aclanthology.org/2021.icon-main.16
DOI:
Bibkey:
Cite (ACL):
Ratnesh Joshi, Arindam Chatterjee, and Asif Ekbal. 2021. Towards Explainable Dialogue System: Explaining Intent Classification using Saliency Techniques. In Proceedings of the 18th International Conference on Natural Language Processing (ICON), pages 120–127, National Institute of Technology Silchar, Silchar, India. NLP Association of India (NLPAI).
Cite (Informal):
Towards Explainable Dialogue System: Explaining Intent Classification using Saliency Techniques (Joshi et al., ICON 2021)
Copy Citation:
PDF:
https://aclanthology.org/2021.icon-main.16.pdf
Data
ATIS