Identifying User-Inference-Dependent Out-of-Character Behavior in LLM-Based Role-Playing Agents

Shinji Iwata, Shiki Sato, Jun Baba, Koya Ihara, Michimasa Inaba


Abstract
For the practical deployment of the Role-Playing Agents with Large Language Models, prior work has mainly focused on suppressing out-of-character behavior relative to explicitly provided character information such as personas. However, as shown in psychological research, out-of-character behavior can arise not only from contradictions with explicit information but also from mismatches with character images implicitly inferred by users. Modeling user-inference-dependent out-of-character behavior is a challenging task, because it is unclear what kinds of inferences users make from existing information. To tackle this problem, we construct a dataset of 480 dialogue scenarios and collect 2,152 descriptions from users familiar with those titles. Using cases that annotators judged inappropriate despite the absence of contradiction with existing settings, we derive a 12-category typology, organized into general, title-specific, and individual-specific groups. We also conduct an exploratory evaluation of category-guided feedback, finding no consistent overall advantage over standard self-refinement but possible benefits for some characters. Our shareable code and dataset are at https://github.com/CyberAgentAILab/User-Inference-OOC.
Anthology ID:
2026.aacl-long.220
Volume:
Proceedings of the 15th International Joint Conference on Natural Language Processing and the 5th Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics (Volume 1: Long Papers)
Month:
November
Year:
2026
Address:
Hengqin, China
Editors:
Yue Zhang, Ruifeng Xu, Ayu Purwarianti, Tong Rong, Jun Suzuki
Venues:
AACL | IJCNLP
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
4927–4943
Language:
URL:
https://aclanthology.org/2026.aacl-long.220/
DOI:
Bibkey:
Cite (ACL):
Shinji Iwata, Shiki Sato, Jun Baba, Koya Ihara, and Michimasa Inaba. 2026. Identifying User-Inference-Dependent Out-of-Character Behavior in LLM-Based Role-Playing Agents. In Proceedings of the 15th International Joint Conference on Natural Language Processing and the 5th Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics (Volume 1: Long Papers), pages 4927–4943, Hengqin, China. Association for Computational Linguistics.
Cite (Informal):
Identifying User-Inference-Dependent Out-of-Character Behavior in LLM-Based Role-Playing Agents (Iwata et al., AACL-IJCNLP 2026)
Copy Citation:
PDF:
https://aclanthology.org/2026.aacl-long.220.pdf