STIR: Self-Distilled Latent Tools for Dynamic Control of LLM Reasoning

Zhenning Shi, Yijia Zhu, Junhan Shi, Xun Zhang, Lei Wang, Congcong Miao


Abstract
Chain-of-thought reasoning improves large language models by spending more tokens at test time, but this couples accuracy to generation length. Activation steering offers a lightweight alternative, yet most steering methods apply a fixed vector throughout generation and cannot adapt to different stages of a multi-step solution. We propose STIR (Self-Distilled Tools for Internal Reasoning), a test-time framework that builds a compact library of state-conditioned correction vectors from the model’s own successful and failed rollouts. At inference time, STIR retrieves candidate corrections at structural reasoning checkpoints, validates them with a short lookahead probe, and abstains when the current trajectory already matches a high-quality state. Across six mathematical and knowledge-intensive reasoning benchmarks, STIR improves average accuracy by 1.9% to 7.5% over Vanilla decoding while reducing average token consumption by up to 35%. These results suggest that selective latent interventions can improve the accuracy-cost trade-off of reasoning without forcing every problem to generate longer chains. Our code is available at https://github.com/sznnzs/STIR.
Anthology ID:
2026.emnlp-main.2278
Volume:
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Month:
October
Year:
2026
Address:
Budapest, Hungary
Editors:
Hung-Yi Lee, Sunipa Dev, Jessy Li, Desmond Elliott
Venue:
EMNLP
SIG:
SIGDAT
Publisher:
Association for Computational Linguistics
Note:
Pages:
51814–51829
Language:
URL:
https://aclanthology.org/2026.emnlp-main.2278/
DOI:
Bibkey:
Cite (ACL):
Zhenning Shi, Yijia Zhu, Junhan Shi, Xun Zhang, Lei Wang, and Congcong Miao. 2026. STIR: Self-Distilled Latent Tools for Dynamic Control of LLM Reasoning. In Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing, pages 51814–51829, Budapest, Hungary. Association for Computational Linguistics.
Cite (Informal):
STIR: Self-Distilled Latent Tools for Dynamic Control of LLM Reasoning (Shi et al., EMNLP 2026)
Copy Citation:
PDF:
https://aclanthology.org/2026.emnlp-main.2278.pdf