Ting Mao
Author directory2026
Analyzing and Steering Reflection in Large Reasoning Models: A Difficulty-Aware Geometric Approach
Ge Teng | Chen Shen | Ting Mao | Xiang Tian | Xuesong Liu | Rongxin Jiang | Yaowu Chen | Jieping Ye
Findings of the Association for Computational Linguistics: EMNLP 2026
Ge Teng | Chen Shen | Ting Mao | Xiang Tian | Xuesong Liu | Rongxin Jiang | Yaowu Chen | Jieping Ye
Findings of the Association for Computational Linguistics: EMNLP 2026
Large Reasoning Models (LRMs) have demonstrated exceptional capabilities in complex problem-solving, largely driven by emergent reflection and backtracking mechanisms. However, understanding these mechanisms through mechanistic interpretability remains challenging. Current sparsity-based approaches rely heavily on heuristic keywords (e.g., "wait") and focus on isolated features, lacking a holistic view of how these features collaborate. In this paper, we propose a novel interpretability framework that bridges the microscopic and macroscopic analysis of Sparse Auto-Encoder (SAE) features. By leveraging an auxiliary LRM to robustly annotate reasoning trajectories, we introduce a comprehensive "Reflection Score" to quantify a feature’s relevance to reflection without keyword bias. Our geometric analysis of these features reveals that LRMs encode reflection not as discrete, isolated vectors, but within continuous regions of the hidden state space. Furthermore, we uncover an intrinsic, difficulty-aware reflection representation mechanism: the model dynamically activates sub-regions based on problem complexity. Building upon these insights, we introduce a static, difficulty-informed steering strategy that suppresses superficial, simple-problem oriented reflection features during inference. Comprehensive evaluations across three LRMs and challenging reasoning benchmarks demonstrate that our intervention simultaneously improves reasoning accuracy by an average of 2.25% and reduces token consumption by 5.3%. Our findings provide a deeper geometric understanding of LRM reflection and offer an interpretable, scalable approach to optimizing LRM reasoning.