Collaborative Data Relabeling for Robust and Diverse Voice Apps Recommendation in Intelligent Personal Assistants

Qian Hu, Thahir Mohamed, Zheng Gao, Xibin Gao, Radhika Arava, Xiyao Ma, Mohamed AbdelHady


Abstract
Intelligent personal assistants (IPAs) such as Amazon Alexa, Google Assistant and Apple Siri extend their built-in capabilities by supporting voice apps developed by third-party developers. Sometimes the smart assistant is not able to successfully respond to user voice commands (aka utterances). There are many reasons including automatic speech recognition (ASR) error, natural language understanding (NLU) error, routing utterances to an irrelevant voice app or simply that the user is asking for a capability that is not supported yet. The failure to handle a voice command leads to customer frustration. In this paper, we introduce a fallback skill recommendation system to suggest a voice app to a customer for an unhandled voice command. One of the prominent challenges of developing a skill recommender system for IPAs is partial observation. To solve the partial observation problem, we propose collaborative data relabeling (CDR) method. In addition, CDR also improves the diversity of the recommended skills. We evaluate the proposed method both offline and online. The offline evaluation results show that the proposed system outperforms the baselines. The online A/B testing results show significant gain of customer experience metrics.
Anthology ID:
2021.nlp4convai-1.11
Volume:
Proceedings of the 3rd Workshop on Natural Language Processing for Conversational AI
Month:
November
Year:
2021
Address:
Online
Venues:
EMNLP | NLP4ConvAI
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
113–119
Language:
URL:
https://aclanthology.org/2021.nlp4convai-1.11
DOI:
10.18653/v1/2021.nlp4convai-1.11
Bibkey:
Cite (ACL):
Qian Hu, Thahir Mohamed, Zheng Gao, Xibin Gao, Radhika Arava, Xiyao Ma, and Mohamed AbdelHady. 2021. Collaborative Data Relabeling for Robust and Diverse Voice Apps Recommendation in Intelligent Personal Assistants. In Proceedings of the 3rd Workshop on Natural Language Processing for Conversational AI, pages 113–119, Online. Association for Computational Linguistics.
Cite (Informal):
Collaborative Data Relabeling for Robust and Diverse Voice Apps Recommendation in Intelligent Personal Assistants (Hu et al., NLP4ConvAI 2021)
Copy Citation:
PDF:
https://aclanthology.org/2021.nlp4convai-1.11.pdf