Detecting Ambiguous Utterances in an Intelligent Assistant

Satoshi Akasaki, Manabu Sassano


Abstract
In intelligent assistants that perform both chatting and tasks through dialogue, like Siri and Alexa, users often make ambiguous utterances such as “I’m hungry” or “I have a headache,” which can be interpreted as either chat or task intents. Naively determining these intents can lead to mismatched responses, spoiling the user experience. Therefore, it is desirable to determine the ambiguity of user utterances. We created a dataset from an actual intelligent assistant via crowdsourcing and analyzed tendencies of ambiguous utterances. Using this labeled data of chat, task, and ambiguous intents, we developed a supervised intent classification model. To detect ambiguous utterances robustly, we propose feeding sentence embeddings developed from microblogs and search logs with a self-attention mechanism. Experiments showed that our model outperformed two baselines, including a strong LLM-based one. We will release the dataset.
Anthology ID:
2024.emnlp-industry.28
Volume:
Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing: Industry Track
Month:
November
Year:
2024
Address:
Miami, Florida, US
Editors:
Franck Dernoncourt, Daniel Preoţiuc-Pietro, Anastasia Shimorina
Venue:
EMNLP
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
386–394
Language:
URL:
https://aclanthology.org/2024.emnlp-industry.28
DOI:
10.18653/v1/2024.emnlp-industry.28
Bibkey:
Cite (ACL):
Satoshi Akasaki and Manabu Sassano. 2024. Detecting Ambiguous Utterances in an Intelligent Assistant. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing: Industry Track, pages 386–394, Miami, Florida, US. Association for Computational Linguistics.
Cite (Informal):
Detecting Ambiguous Utterances in an Intelligent Assistant (Akasaki & Sassano, EMNLP 2024)
Copy Citation:
PDF:
https://aclanthology.org/2024.emnlp-industry.28.pdf
Poster:
 2024.emnlp-industry.28.poster.pdf