Xiang Tian
Author directoryOther people with similar names: Xiang Tian
2026
Analyzing and Steering Reflection in Large Reasoning Models: A Difficulty-Aware Geometric Approach
Ge Teng | Chen Shen | Ting Mao | Xiang Tian | Xuesong Liu | Rongxin Jiang | Yaowu Chen | Jieping Ye
Findings of the Association for Computational Linguistics: EMNLP 2026
Ge Teng | Chen Shen | Ting Mao | Xiang Tian | Xuesong Liu | Rongxin Jiang | Yaowu Chen | Jieping Ye
Findings of the Association for Computational Linguistics: EMNLP 2026
Large Reasoning Models (LRMs) have demonstrated exceptional capabilities in complex problem-solving, largely driven by emergent reflection and backtracking mechanisms. However, understanding these mechanisms through mechanistic interpretability remains challenging. Current sparsity-based approaches rely heavily on heuristic keywords (e.g., "wait") and focus on isolated features, lacking a holistic view of how these features collaborate. In this paper, we propose a novel interpretability framework that bridges the microscopic and macroscopic analysis of Sparse Auto-Encoder (SAE) features. By leveraging an auxiliary LRM to robustly annotate reasoning trajectories, we introduce a comprehensive "Reflection Score" to quantify a feature’s relevance to reflection without keyword bias. Our geometric analysis of these features reveals that LRMs encode reflection not as discrete, isolated vectors, but within continuous regions of the hidden state space. Furthermore, we uncover an intrinsic, difficulty-aware reflection representation mechanism: the model dynamically activates sub-regions based on problem complexity. Building upon these insights, we introduce a static, difficulty-informed steering strategy that suppresses superficial, simple-problem oriented reflection features during inference. Comprehensive evaluations across three LRMs and challenging reasoning benchmarks demonstrate that our intervention simultaneously improves reasoning accuracy by an average of 2.25% and reduces token consumption by 5.3%. Our findings provide a deeper geometric understanding of LRM reflection and offer an interpretable, scalable approach to optimizing LRM reasoning.
2025
Tracing and Dissecting How LLMs Recall Factual Knowledge for Real World Questions
Yiqun Wang | Chaoqun Wan | Sile Hu | Yonggang Zhang | Xiang Tian | Yaowu Chen | Xu Shen | Jieping Ye
Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
Yiqun Wang | Chaoqun Wan | Sile Hu | Yonggang Zhang | Xiang Tian | Yaowu Chen | Xu Shen | Jieping Ye
Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
Recent advancements in large language models (LLMs) have shown promising ability to perform commonsense reasoning, bringing machines closer to human-like understanding. However, deciphering the internal reasoning processes of LLMs remains challenging due to the complex interdependencies among generated tokens, especially in practical question-answering. In this study, we introduce a two-dimensional analysis framework—comprising token back-tracing and individual token decoding—to uncover how LLMs conduct factual knowledge recall. Through explanatory analysis of three typical reasoning datasets, we identify a consistent three-phase pattern: Subject Augmentation and Broadcasting, Object Retrieval and Reranking, and Conclusion Fusion and Generation. Our findings reveal that LLMs do not lack relevant knowledge but struggle to select the most accurate information based on context during the retrieval and rerank phase. Leveraging these findings, we apply representation engineering and selective fine-tuning to target specific modules responsible for retrieval and rerank errors. Experimental results show large improvements in response accuracy for both in-domain and out-of-domain settings, validating the rationality of the interpreting result.