On the Controllability of Large Language Models for Dialogue Interaction

Nicolas Wagner, Stefan Ultes


Abstract
This paper investigates the enhancement of Dialogue Systems by integrating the creative capabilities of Large Language Models. While traditional Dialogue Systems focus on understanding user input and selecting appropriate system actions, Language Models excel at generating natural language text based on prompts. Therefore, we propose to improve controllability and coherence of interactions by guiding a Language Model with control signals that enable explicit control over the system behaviour. To address this, we tested and evaluated our concept in 815 conversations with over 3600 dialogue exchanges on a dataset. Our experiment examined the quality of generated system responses using two strategies: An unguided strategy where task data was provided to the models, and a controlled strategy in which a simulated Dialogue Controller provided appropriate system actions. The results show that the average BLEU score and the classification of dialogue acts improved in the controlled Natural Language Generation.
Anthology ID:
2024.sigdial-1.19
Volume:
Proceedings of the 25th Annual Meeting of the Special Interest Group on Discourse and Dialogue
Month:
September
Year:
2024
Address:
Kyoto, Japan
Editors:
Tatsuya Kawahara, Vera Demberg, Stefan Ultes, Koji Inoue, Shikib Mehri, David Howcroft, Kazunori Komatani
Venue:
SIGDIAL
SIG:
SIGDIAL
Publisher:
Association for Computational Linguistics
Note:
Pages:
216–221
Language:
URL:
https://aclanthology.org/2024.sigdial-1.19
DOI:
Bibkey:
Cite (ACL):
Nicolas Wagner and Stefan Ultes. 2024. On the Controllability of Large Language Models for Dialogue Interaction. In Proceedings of the 25th Annual Meeting of the Special Interest Group on Discourse and Dialogue, pages 216–221, Kyoto, Japan. Association for Computational Linguistics.
Cite (Informal):
On the Controllability of Large Language Models for Dialogue Interaction (Wagner & Ultes, SIGDIAL 2024)
Copy Citation:
PDF:
https://aclanthology.org/2024.sigdial-1.19.pdf