Haoxiang Gao

Author directory

2026

As large language model (LLM) chatbots become embedded in education, researchers need scalable ways to analyze what learners are actually doing in student-chatbot interactions. To address this challenge, we propose a learning theory-informed annotation framework for learner intent and engagement in educational chatbot dialogues, operationalizing solution seeking, knowledge seeking, cognitive engagement, and mental model expression through a hierarchical decision process, and apply it manually to a dataset of 2,393 learner-chatbot interactions (408 double-coded). We then evaluate two LLMs as zero-shot annotators and compare human-human, LLM-LLM, and human-LLM agreement across both intermediate decisions and final intent labels. While the two LLMs exhibit agreement patterns approaching human-human reliability on the final label (human κ = 0.73 vs. LLM κ = 0.67), their agreement with human reference labels is substantially lower (κ ≤ 0.53), indicating that automated annotation errors cannot be understood from aggregate scores alone. Through exploratory qualitative coding of disagreement cases, we find that divergences can arise from ambiguous learner intent, competing valid interpretations, and human annotation variability, rather than model failure alone.

2025

Recent advances in large language models (LLMs) have sparked growing interest in agentic workflows—structured sequences of LLM invocations designed to solve complex tasks. However, existing approaches often rely on static templates or manually designed workflows, which limit adaptability to diverse tasks and hinder scalability. We propose AdaptFlow, a natural language-based meta-learning framework inspired by model-agnostic meta-learning (MAML). AdaptFlow uses a bi-level optimization process: the inner loop performs task-specific adaptation via LLM-generated feedback, while the outer loop consolidates these refinements into a shared, generalizable initialization. Evaluated across question answering, code generation, and mathematical reasoning benchmarks, AdaptFlow consistently outperforms both manually crafted and automatically searched baselines, achieving state-of-the-art results with strong generalization across tasks and models.