Weida Liang
Author directory2026
From Harm to Help: Turning Reasoning In-Context Demos into Assets for Reasoning LMs
Weida Liang | Zihang Fu | Haonan Wang | Zheng Nie | Yifan Zhang | Yao Tong | Tongyao Zhu | Hao Jiang | Chuang Li | Jiaying Wu | Kenji Kawaguchi
Findings of the Association for Computational Linguistics: EMNLP 2026
Weida Liang | Zihang Fu | Haonan Wang | Zheng Nie | Yifan Zhang | Yao Tong | Tongyao Zhu | Hao Jiang | Chuang Li | Jiaying Wu | Kenji Kawaguchi
Findings of the Association for Computational Linguistics: EMNLP 2026
Recent reasoning LLMs (RLMs), especially those trained with verifier-based reinforcement learning, often perform worse with few-shot CoT than with direct answering. We revisit this paradox using high-quality reasoning traces from DeepSeek-R1 as demonstrations and find that adding more exemplars consistently degrades accuracy, even when demonstrations are optimal. A detailed analysis reveals two mechanisms behind this decline: (i) *semantic misguidance*, where high textual similarity leads the model to treat the target as the same as the exemplar and to copy intermediate steps verbatim; and (ii) *strategy transfer failure*, where the model struggles to extract useful reasoning strategies and apply them to target questions. Guided by these findings, we introduce Insight-to-Solve (I2S), which converts demonstrations into target-conditioned insights before solving, and I2S+, which adds iterative refinement. Across open- and closed-source models, both variants outperform direct answering and sequential test-time baselines. On a frozen 500-problem Omni-MATH transfer set, I2S+ improves R1-Distill-Qwen-7B and Qwen3-14B by +4.2 and +4.0 points, respectively. The same prompt stack also transfers to GPT-4.1, o1-mini, and R1-Distill-Llama-8B, showing that reasoning demonstrations can be converted from liabilities into useful test-time guidance.