Intent Mining from past conversations for Conversational Agent

Ajay Chatterjee, Shubhashis Sengupta


Abstract
Conversational systems are of primary interest in the AI community. Organizations are increasingly using chatbot to provide round-the-clock support and to increase customer engagement. Many commercial bot building frameworks follow a standard approach that requires one to build and train an intent model to recognize user input. These frameworks require a collection of user utterances and corresponding intent to train an intent model. Collecting a substantial coverage of training data is a bottleneck in the bot building process. In cases where past conversation data is available, the cost of labeling hundreds of utterances with intent labels is time-consuming and laborious. In this paper, we present an intent discovery framework that can mine a vast amount of conversational logs and to generate labeled data sets for training intent models. We have introduced an extension to the DBSCAN algorithm and presented a density-based clustering algorithm ITER-DBSCAN for unbalanced data clustering. Empirical evaluation on one conversation dataset, six different intent dataset, and one short text clustering dataset show the effectiveness of our hypothesis.
Anthology ID:
2020.coling-main.366
Volume:
Proceedings of the 28th International Conference on Computational Linguistics
Month:
December
Year:
2020
Address:
Barcelona, Spain (Online)
Editors:
Donia Scott, Nuria Bel, Chengqing Zong
Venue:
COLING
SIG:
Publisher:
International Committee on Computational Linguistics
Note:
Pages:
4140–4152
Language:
URL:
https://aclanthology.org/2020.coling-main.366
DOI:
10.18653/v1/2020.coling-main.366
Bibkey:
Cite (ACL):
Ajay Chatterjee and Shubhashis Sengupta. 2020. Intent Mining from past conversations for Conversational Agent. In Proceedings of the 28th International Conference on Computational Linguistics, pages 4140–4152, Barcelona, Spain (Online). International Committee on Computational Linguistics.
Cite (Informal):
Intent Mining from past conversations for Conversational Agent (Chatterjee & Sengupta, COLING 2020)
Copy Citation:
PDF:
https://aclanthology.org/2020.coling-main.366.pdf
Code
 ajaychatterjee/IntentMining