Haoxiang Gao
Author directory2026
When Humans and LLMs Disagree: Understanding Human-LLM Disagreement in Learner Intent Annotation for Educational Chatbot Dialogues
Isabel Schorr | Lisa Bardach | Haoxiang Gao | Enkelejda Kasneci
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Isabel Schorr | Lisa Bardach | Haoxiang Gao | Enkelejda Kasneci
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
As large language model (LLM) chatbots become embedded in education, researchers need scalable ways to analyze what learners are actually doing in student-chatbot interactions. To address this challenge, we propose a learning theory-informed annotation framework for learner intent and engagement in educational chatbot dialogues, operationalizing solution seeking, knowledge seeking, cognitive engagement, and mental model expression through a hierarchical decision process, and apply it manually to a dataset of 2,393 learner-chatbot interactions (408 double-coded). We then evaluate two LLMs as zero-shot annotators and compare human-human, LLM-LLM, and human-LLM agreement across both intermediate decisions and final intent labels. While the two LLMs exhibit agreement patterns approaching human-human reliability on the final label (human κ = 0.73 vs. LLM κ = 0.67), their agreement with human reference labels is substantially lower (κ ≤ 0.53), indicating that automated annotation errors cannot be understood from aggregate scores alone. Through exploratory qualitative coding of disagreement cases, we find that divergences can arise from ambiguous learner intent, competing valid interpretations, and human annotation variability, rather than model failure alone.
2025
AdaptFlow: Adaptive Workflow Optimization via Meta-Learning
Runchuan Zhu | Bowen Jiang | Lingrui Mei | Fangkai Yang | Lu Wang | Haoxiang Gao | Fengshuo Bai | Pu Zhao | Qingwei Lin | Saravan Rajmohan | Dongmei Zhang
Findings of the Association for Computational Linguistics: EMNLP 2025
Runchuan Zhu | Bowen Jiang | Lingrui Mei | Fangkai Yang | Lu Wang | Haoxiang Gao | Fengshuo Bai | Pu Zhao | Qingwei Lin | Saravan Rajmohan | Dongmei Zhang
Findings of the Association for Computational Linguistics: EMNLP 2025
Recent advances in large language models (LLMs) have sparked growing interest in agentic workflows—structured sequences of LLM invocations designed to solve complex tasks. However, existing approaches often rely on static templates or manually designed workflows, which limit adaptability to diverse tasks and hinder scalability. We propose AdaptFlow, a natural language-based meta-learning framework inspired by model-agnostic meta-learning (MAML). AdaptFlow uses a bi-level optimization process: the inner loop performs task-specific adaptation via LLM-generated feedback, while the outer loop consolidates these refinements into a shared, generalizable initialization. Evaluated across question answering, code generation, and mathematical reasoning benchmarks, AdaptFlow consistently outperforms both manually crafted and automatically searched baselines, achieving state-of-the-art results with strong generalization across tasks and models.