Erin Walker

Author directory

2026

Most state-of-the-art sign language models are trained on interpreter or isolated vocabulary data, which overlooks the variability that characterizes natural dialogue. However, human communication dynamically adapts to contexts and interlocutors through spatiotemporal changes and articulation style. This specifically manifests itself in educational settings, where novel vocabularies are used by teachers, and students. To address this gap, we collect a motion capture dataset of American Sign Language (ASL) STEM (Science, Technology, Engineering, and Mathematics) dialogue that enables quantitative comparison between dyadic interactive signing, solo signed lecture, and interpreted articles. Using continuous kinematic features, we disentangle dialogue-specific entrainment from individual effort reduction and show spatiotemporal changes across repeated mentions of STEM terms. On average, dialogue signs are 24.6%-44.6% shorter in duration than the isolated signs, and show significant reductions absent in monologue contexts. Finally, we evaluate sign embedding models on their ability to recognize STEM signs and approximate how entrained the participants become over time. Our study bridges linguistic analysis and computational modeling to understand how pragmatics shape sign articulation and its representation in sign language technologies.
Multiparty tutoring policies must regulate turn-taking and social attention, yet collecting the interaction data needed for reinforcement learning is expensive in educational settings. We introduce an LLM-based user simulator for triadic robot–student dialogue that represents gaze and gesture as inline stage directions. Combined with phase-tagged prompting (which delineates conversational structure) and per-turn persona reinforcement, this design allows the LLMs to generate interactions that preserve key discourse structure and controllable participation dynamics. We validate six simulator variants against a real student–robot corpus (12 student pairs) using metrics for semantic similarity, cue obedience, and persona consistency. While the simulator reaches strong semantic agreement with real transcripts, it also reveals a measurable sim-to-real style gap (in verbosity and grammatical complexity). The simulator enables the creation of an RL environment where the tutor selects factorized gaze and reference actions, learning to interact with simulated students. We compare online training (APPO) to offline learning (DQN). Off-policy evaluation on real logs shows that offline policies trained purely from synthetic data exceed online-trained policies under a doubly-robust estimator, and a verbal-only ablation yields unstable OPE due to heavy-tailed importance weights. These results support a scalable workflow for prototyping socially grounded multiparty tutoring policies while relaying where real-world coverage and OPE reliability remain bottlenecks.

2025

Large language models (LLMs) have emerged as powerful tools for developing educational systems. While previous studies have explored modeling student mistakes, a critical gap remains in understanding whether LLMs can generate correct solutions that represent student responses to free-response problems. In this paper, we compare the distribution of solutions produced by four LLMs (one proprietary, two open-sourced general, and one open-sourced math models) with various sampling and prompting techniques and those generated by students, using conversations where students teach math problems to a conversational robot. Our study reveals discrepancies between the correct solutions produced by LLMs and by students. We discuss the practical implications of these findings for the design and evaluation of LLM-supported educational systems.

2022

Speakers build rapport in the process of aligning conversational behaviors with each other. Rapport engendered with a teachable agent while instructing domain material has been shown to promote learning. Past work on lexical alignment in the field of education suffers from limitations in both the measures used to quantify alignment and the types of interactions in which alignment with agents has been studied. In this paper, we apply alignment measures based on a data-driven notion of shared expressions (possibly composed of multiple words) and compare alignment in one-on-one human-robot (H-R) interactions with the H-R portions of collaborative human-human-robot (H-H-R) interactions. We find that students in the H-R setting align with a teachable robot more than in the H-H-R setting and that the relationship between lexical alignment and rapport is more complex than what is predicted by previous theoretical and empirical work.