PersonaCLR: Evaluation Model for Persona Characteristics via Contrastive Learning of Linguistic Style Representation

Michimasa Inaba


Abstract
Persona-aware dialogue systems can improve the consistency of the system’s responses, users’ trust and user enjoyment. Filtering nonpersona-like utterances is important for constructing persona-aware dialogue systems. This paper presents the PersonaCLR model for capturing a given utterance’s intensity of persona characteristics. We trained the model with contrastive learning based on the sameness of the utterances’ speaker. Contrastive learning enables PersonaCLR to evaluate the persona characteristics of a given utterance, even if the target persona is not included in training data. For training and evaluating our model, we also constructed a new dataset of 2,155 character utterances from 100 Japanese online novels. Experimental results indicated that our model outperforms existing methods and a strong baseline using a large language model. Our source code, pre-trained model, and dataset are available at https://github.com/1never/PersonaCLR.
Anthology ID:
2024.sigdial-1.58
Volume:
Proceedings of the 25th Annual Meeting of the Special Interest Group on Discourse and Dialogue
Month:
September
Year:
2024
Address:
Kyoto, Japan
Editors:
Tatsuya Kawahara, Vera Demberg, Stefan Ultes, Koji Inoue, Shikib Mehri, David Howcroft, Kazunori Komatani
Venue:
SIGDIAL
SIG:
SIGDIAL
Publisher:
Association for Computational Linguistics
Note:
Pages:
674–685
Language:
URL:
https://aclanthology.org/2024.sigdial-1.58
DOI:
Bibkey:
Cite (ACL):
Michimasa Inaba. 2024. PersonaCLR: Evaluation Model for Persona Characteristics via Contrastive Learning of Linguistic Style Representation. In Proceedings of the 25th Annual Meeting of the Special Interest Group on Discourse and Dialogue, pages 674–685, Kyoto, Japan. Association for Computational Linguistics.
Cite (Informal):
PersonaCLR: Evaluation Model for Persona Characteristics via Contrastive Learning of Linguistic Style Representation (Inaba, SIGDIAL 2024)
Copy Citation:
PDF:
https://aclanthology.org/2024.sigdial-1.58.pdf