ARTA: Collection and Classification of Ambiguous Requests and Thoughtful Actions

Shohei Tanaka, Koichiro Yoshino, Katsuhito Sudoh, Satoshi Nakamura


Abstract
Human-assisting systems such as dialogue systems must take thoughtful, appropriate actions not only for clear and unambiguous user requests, but also for ambiguous user requests, even if the users themselves are not aware of their potential requirements. To construct such a dialogue agent, we collected a corpus and developed a model that classifies ambiguous user requests into corresponding system actions. In order to collect a high-quality corpus, we asked workers to input antecedent user requests whose pre-defined actions could be regarded as thoughtful. Although multiple actions could be identified as thoughtful for a single user request, annotating all combinations of user requests and system actions is impractical. For this reason, we fully annotated only the test data and left the annotation of the training data incomplete. In order to train the classification model on such training data, we applied the positive/unlabeled (PU) learning method, which assumes that only a part of the data is labeled with positive examples. The experimental results show that the PU learning method achieved better performance than the general positive/negative (PN) learning method to classify thoughtful actions given an ambiguous user request.
Anthology ID:
2021.sigdial-1.9
Volume:
Proceedings of the 22nd Annual Meeting of the Special Interest Group on Discourse and Dialogue
Month:
July
Year:
2021
Address:
Singapore and Online
Editors:
Haizhou Li, Gina-Anne Levow, Zhou Yu, Chitralekha Gupta, Berrak Sisman, Siqi Cai, David Vandyke, Nina Dethlefs, Yan Wu, Junyi Jessy Li
Venue:
SIGDIAL
SIG:
SIGDIAL
Publisher:
Association for Computational Linguistics
Note:
Pages:
77–88
Language:
URL:
https://aclanthology.org/2021.sigdial-1.9
DOI:
10.18653/v1/2021.sigdial-1.9
Bibkey:
Cite (ACL):
Shohei Tanaka, Koichiro Yoshino, Katsuhito Sudoh, and Satoshi Nakamura. 2021. ARTA: Collection and Classification of Ambiguous Requests and Thoughtful Actions. In Proceedings of the 22nd Annual Meeting of the Special Interest Group on Discourse and Dialogue, pages 77–88, Singapore and Online. Association for Computational Linguistics.
Cite (Informal):
ARTA: Collection and Classification of Ambiguous Requests and Thoughtful Actions (Tanaka et al., SIGDIAL 2021)
Copy Citation:
PDF:
https://aclanthology.org/2021.sigdial-1.9.pdf
Video:
 https://www.youtube.com/watch?v=Y4OAaQzoIhA
Code
 ahclab/arta_corpus