Retrieving Responses Useful as References in Counseling via LLM-Generated Structurally Analogous Dialogues

Yu Nakagawa, Nozomu Ikeda, Kotaro Funakoshi, Manabu Okumura


Abstract
Retrieving past similar dialogues to assist response generation in the present context is a promising approach to addressing the shortage of highly experienced professionals in various domains, such as user support and mental health counseling. However, unlike traditional retrieval, this task requires finding dialogue histories that lead to useful subsequent responses—a property we define as referenceability—rather than relying solely on superficial topic matching. Learning this property directly is difficult due to the impracticality of manually annotating large-scale data. To address this, we propose a framework that leverages large language models to generate pseudo-similar counseling dialogues with strict message-level correspondence to original sessions. We then use these generated dialogues as weak supervision for contrastive learning of an embedding model. By pairing original dialogue histories with their generated counterparts, the model learns to pull together histories that share referenceable subsequent counselor responses, even if their explicit situations differ. Experimental results show that our fine-tuned model significantly improves retrieval performance, increasing Hit@1 from 0.167 to 0.500 and MRR from 0.232 to 0.576. Human evaluations also confirm that the model retrieves more referenceable responses, improving the average score from 2.22 to 2.66.
Anthology ID:
2026.sigdial-1.20
Volume:
Proceedings of the 27th Annual Meeting of the Special Interest Group on Discourse and Dialogue
Month:
August
Year:
2026
Address:
Atlanta, Georgia, USA
Editors:
Jinho D. Choi, Yun-Nung Chen, Kotaro Funakoshi, Ali Emami
Venue:
SIGDIAL
SIG:
SIGDIAL
Publisher:
Association for Computational Linguistics
Note:
Pages:
271–292
Language:
URL:
https://aclanthology.org/2026.sigdial-1.20/
DOI:
Bibkey:
Cite (ACL):
Yu Nakagawa, Nozomu Ikeda, Kotaro Funakoshi, and Manabu Okumura. 2026. Retrieving Responses Useful as References in Counseling via LLM-Generated Structurally Analogous Dialogues. In Proceedings of the 27th Annual Meeting of the Special Interest Group on Discourse and Dialogue, pages 271–292, Atlanta, Georgia, USA. Association for Computational Linguistics.
Cite (Informal):
Retrieving Responses Useful as References in Counseling via LLM-Generated Structurally Analogous Dialogues (Nakagawa et al., SIGDIAL 2026)
Copy Citation:
PDF:
https://aclanthology.org/2026.sigdial-1.20.pdf