Investigating person-specific errors in chat-oriented dialogue systems

Koh Mitsuda, Ryuichiro Higashinaka, Tingxuan Li, Sen Yoshida


Abstract
Creating chatbots to behave like real people is important in terms of believability. Errors in general chatbots and chatbots that follow a rough persona have been studied, but those in chatbots that behave like real people have not been thoroughly investigated. We collected a large amount of user interactions of a generation-based chatbot trained from large-scale dialogue data of a specific character, i.e., target person, and analyzed errors related to that person. We found that person-specific errors can be divided into two types: errors in attributes and those in relations, each of which can be divided into two levels: self and other. The correspondence with an existing taxonomy of errors was also investigated, and person-specific errors that should be addressed in the future were clarified.
Anthology ID:
2022.acl-short.50
Volume:
Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)
Month:
May
Year:
2022
Address:
Dublin, Ireland
Editors:
Smaranda Muresan, Preslav Nakov, Aline Villavicencio
Venue:
ACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
464–469
Language:
URL:
https://aclanthology.org/2022.acl-short.50
DOI:
10.18653/v1/2022.acl-short.50
Bibkey:
Cite (ACL):
Koh Mitsuda, Ryuichiro Higashinaka, Tingxuan Li, and Sen Yoshida. 2022. Investigating person-specific errors in chat-oriented dialogue systems. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 464–469, Dublin, Ireland. Association for Computational Linguistics.
Cite (Informal):
Investigating person-specific errors in chat-oriented dialogue systems (Mitsuda et al., ACL 2022)
Copy Citation:
PDF:
https://aclanthology.org/2022.acl-short.50.pdf