Tova Radhe
Author directory2026
The Hidden Cost of Personality: How Conditioning Methods Disrupt Gesture Generation in Small Language Models
Tova Radhe | Alessio Galatolo
Proceedings of the 1st Workshop on Multimodal Interaction in Face-to-Face Dialogue
Tova Radhe | Alessio Galatolo
Proceedings of the 1st Workshop on Multimodal Interaction in Face-to-Face Dialogue
Personality is a key factor in human–agent and human-robot interaction, influencing user perception, trust, and engagement. This work investigates methods for inducing Big Five personality traits in (open) language models, with a focus on smaller variants as more easily deployable in real-time agents and robotic systems. We compare prompt-based conditioning, parameter-efficient fine-tuning, and activation steering, focusing on their behaviour in multi-turn dialogue and embodied gesture generation. In particular, we investigate personality drift across turns and non-verbal behaviour generation through direct function calls.
We find that classifier-assigned trait alignment differs little across methods, with no consistent evidence of trait drift over multi-turn interaction. However, conditioning method significantly affects dialogue quality and robustness, with steering and fine-tuning introducing artefacts and instability. Interestingly, the more consequential differences emerge in the multimodal channel. Conditioning method, not intended personality, is the dominant driver of gesture behaviour, reshaping gesture diversity, tool-use reliability, and output-gesture congruence. Our findings suggest that personality conditioning should be evaluated jointly across verbal and non-verbal channels, and that method choice is itself a design decision shaping the expressiveness and reliability of embodied conversational agents.
We find that classifier-assigned trait alignment differs little across methods, with no consistent evidence of trait drift over multi-turn interaction. However, conditioning method significantly affects dialogue quality and robustness, with steering and fine-tuning introducing artefacts and instability. Interestingly, the more consequential differences emerge in the multimodal channel. Conditioning method, not intended personality, is the dominant driver of gesture behaviour, reshaping gesture diversity, tool-use reliability, and output-gesture congruence. Our findings suggest that personality conditioning should be evaluated jointly across verbal and non-verbal channels, and that method choice is itself a design decision shaping the expressiveness and reliability of embodied conversational agents.