Thomas Nguyen
Author directory2026
Evaluation of Motivational Interviewing Counsellors with Task-Aware Multi-Stage LLM-Based Simulated Clients
Jiading Zhu | Xinyu Cindy Wang | Thomas Nguyen | Yan Qing Lee | Osnat C. Melamed | Peter Selby | Jonathan Rose
Proceedings of the 3rd Workshop on Language Understanding in the Human-Machine Era (LUHME 2026)
Jiading Zhu | Xinyu Cindy Wang | Thomas Nguyen | Yan Qing Lee | Osnat C. Melamed | Peter Selby | Jonathan Rose
Proceedings of the 3rd Workshop on Language Understanding in the Human-Machine Era (LUHME 2026)
The development and benchmarking of Large Language Model (LLM)-based Motivational Interviewing (MI) counsellors now often rely on LLM-based simulated clients. Prior work on simulated clients, however, has not aligned with the specific *tasks* fundamental to the MI therapy approach. A key task is *evoking*, in which the counsellor first elicits the client’s ambivalence and then strengthens the client’s motivation for change. We present **Evoke-Sim**, a task-aware, multi-stage LLM-based client simulation framework for evaluating MI counsellors in smoking cessation, designed specifically for the *evoking* MI task. Evoke-Sim employs structured client profiles, an evoking-specific three-stage conversation flow, and a *reveal* policy that regulates which client profile information might be disclosed at each stage. We show that, compared to existing profile-grounded simulated clients, Evoke-Sim better differentiates counsellors’ task-aware performance under our explicit formalization of the *evoking* task, while reducing non-grounded client statements and premature disclosure of client information.