Robinson Vasquez Ferrer
Author directory2026
The Error Is the Signal: Misconception Discovery via Step-Error Annotation Clustering
Robinson Vasquez Ferrer | Mrinmaya Sachan | Shashank Sonkar
Proceedings of the 15th International Joint Conference on Natural Language Processing and the 5th Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics (Volume 1: Long Papers)
Robinson Vasquez Ferrer | Mrinmaya Sachan | Shashank Sonkar
Proceedings of the 15th International Joint Conference on Natural Language Processing and the 5th Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics (Volume 1: Long Papers)
Discovering student misconceptions is a key challenge in intelligent tutoring systems, where identifying recurring error-producing patterns can support targeted feedback and adaptive practice. In settings without a predefined misconception taxonomy, clustering is a natural approach for discovering such patterns at scale. Yet clustering is only as good as its representation: full student records contain problem wording, numerical context, and solution-specific details that embeddings can conflate with the student’s error, obscuring the misconception behind the wrong answer. We construct an LLM step-error-annotation-first pipeline that generates step-error annotations, embeds them with or without problem and student response text, and clusters the resulting representations. On MalruleLib, a 101-class misconception benchmark, clustering step-error annotations alone improves clustering accuracy by 0.22 and normalized mutual information by 0.05 on average over clustering the problem and student response, consistent with annotations capturing recurring error patterns more effectively. On Eedi and unlabeled MalAlgoQA, annotation-based representations show higher average held-out clustering accuracy and internal cluster separation, respectively, than their unannotated baselines. To help experts interpret these clusters, the pipeline then generates candidate misconception descriptors and coherence scores for human triage. Overall, these findings show that input representation substantially affects clustering results and that step-error annotations often improve clustering quality.