Machi Shimmei
Author directory2026
CAT: A CounterArgument Tutoring System based on Logical Structure Analysis
Wenzhi Wang | Thiemo Wambsganss | Chenyang Wang | Paul Reisert | Léane Wettstein | Katja Pott | Kenshi Yamaguchi | Machi Shimmei | Roman Rietsche | Surawat Pothong | Naoya Inoue | Kentaro Inui
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing: System Demonstrations
Wenzhi Wang | Thiemo Wambsganss | Chenyang Wang | Paul Reisert | Léane Wettstein | Katja Pott | Kenshi Yamaguchi | Machi Shimmei | Roman Rietsche | Surawat Pothong | Naoya Inoue | Kentaro Inui
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing: System Demonstrations
Engaging in counterarguments (CAs) has been shown to be a promising approach to effectively support the development of critical thinking skills in educational settings. Developing an argumentation tutoring system that automatically provides personalized feedback on CAs would benefit both teachers and students by mitigating the constraints imposed by limited instructional resources. However, the development of such systems remains underexplored. To address this gap, we develop CAT, a CounterArgument Tutoring system that analyzes the underlying logical structure of CAs and provides tailored feedback accordingly. To evaluate CAT, we conduct a user study with 39 undergraduate students over two weeks in a real classroom, comparing it against an existing argumentation tutoring system as a baseline. The results indicate that CAT supports students in critically evaluating opposing arguments, considering alternative perspectives, and refining their own reasoning. Statistical analysis further shows that CAT elicits more engaged interaction than the baseline system. Moreover, students perceive CAT as easy to use and valuable for improving both their critical thinking and argumentation skills.
2025
FOCUS: A Benchmark for Targeted Socratic Question Generation via Source-Span Grounding
Surawat Pothong | Machi Shimmei | Naoya Inoue | Paul Reisert | Ana Brassard | Wenzhi Wang | Shoichi Naito | Jungmin Choi | Kentaro Inui
Proceedings of the 14th International Joint Conference on Natural Language Processing and the 4th Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics
Surawat Pothong | Machi Shimmei | Naoya Inoue | Paul Reisert | Ana Brassard | Wenzhi Wang | Shoichi Naito | Jungmin Choi | Kentaro Inui
Proceedings of the 14th International Joint Conference on Natural Language Processing and the 4th Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics
We present FOCUS, a benchmark and task setting for Socratic question generation that delivers more informative and targeted feedback to learners. Unlike prior datasets, which rely on broad typologies and lack grounding in the source text, FOCUS introduces a new formulation: each Socratic question is paired with a fine-grained, 11-type typology and an explicit source span from the argument it targets. This design supports clearer, more actionable feedback and facilitates interpretable model evaluation. FOCUS includes 440 annotated instances with moderate partial-match agreement, establishing it as a reliable benchmark. Baseline experiments with representative state-of-the-art models reveal, through detailed error analysis, that even strong models struggle with span selection and context-sensitive categories. An extension study on the LogicClimate dataset further confirms the generalizability of the task and annotation framework. FOCUS sets a new standard for pedagogically grounded and informative Socratic question generation.
Identification of Multiple Logical Interpretations in Counter-Arguments
Wenzhi Wang | Paul Reisert | Shoichi Naito | Naoya Inoue | Machi Shimmei | Surawat Pothong | Jungmin Choi | Kentaro Inui
Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing
Wenzhi Wang | Paul Reisert | Shoichi Naito | Naoya Inoue | Machi Shimmei | Surawat Pothong | Jungmin Choi | Kentaro Inui
Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing
Counter-arguments (CAs) are a good means to improve the critical-thinking skills of learners, especially given that one has to thoroughly consider the logic of initial arguments (IA) when composing their CA. Although several tasks have been created for identifying the logical structure of CAs, no prior work has focused on capturing multiple interpretations of logical structures due to their complexity. In this work, we create CALSA+, a dataset consisting of 134 CAs annotated with 13 logical predicate questions. CALSA+ contains 1,742 instances annotated by 3 expert annotators (5,226 total annotations) with good agreement (Krippendorff 𝛼=0.46). Using CALSA+, we train a model with Reinforcement Learning with Verifiable Rewards (RLVR) to identify multiple logical interpretations and show that models trained with RLVR can perform on par with much bigger proprietary models. Our work is the first to attempt to annotate all the interpretations of logical structure on top of CAs. We publicly release our dataset to facilitate research in CA logical structure identification.