Towards Multi-label Unknown Intent Detection

Yawen Ouyang, Zhen Wu, Xinyu Dai, Shujian Huang, Jiajun Chen


Abstract
Multi-class unknown intent detection has made remarkable progress recently. However, it has a strong assumption that each utterance has only one intent, which does not conform to reality because utterances often have multiple intents. In this paper, we propose a more desirable task, multi-label unknown intent detection, to detect whether the utterance contains the unknown intent, in which each utterance may contain multiple intents. In this task, the unique utterances simultaneously containing known and unknown intents make existing multi-class methods easy to fail. To address this issue, we propose an intuitive and effective method to recognize whether All Intents contained in the utterance are Known (AIK). Our high-level idea is to predict the utterance’s intent number, then check whether the utterance contains the same number of known intents. If the number of known intents is less than the number of intents, it implies that the utterance also contains unknown intents. We benchmark AIK over existing methods, and empirical results suggest that our method obtains state-of-the-art performances. For example, on the MultiWOZ 2.3 dataset, AIK significantly reduces the FPR95 by 12.25% compared to the best baseline.
Anthology ID:
2022.coling-1.52
Volume:
Proceedings of the 29th International Conference on Computational Linguistics
Month:
October
Year:
2022
Address:
Gyeongju, Republic of Korea
Venue:
COLING
SIG:
Publisher:
International Committee on Computational Linguistics
Note:
Pages:
626–635
Language:
URL:
https://aclanthology.org/2022.coling-1.52
DOI:
Bibkey:
Cite (ACL):
Yawen Ouyang, Zhen Wu, Xinyu Dai, Shujian Huang, and Jiajun Chen. 2022. Towards Multi-label Unknown Intent Detection. In Proceedings of the 29th International Conference on Computational Linguistics, pages 626–635, Gyeongju, Republic of Korea. International Committee on Computational Linguistics.
Cite (Informal):
Towards Multi-label Unknown Intent Detection (Ouyang et al., COLING 2022)
Copy Citation:
PDF:
https://aclanthology.org/2022.coling-1.52.pdf
Code
 yawenouyang/aik
Data
MixSNIPsSNIPS