Zhanshuo Dong
Author directory2026
Thinking Without Speaking: Efficient Tree of Thoughts Reasoning via Latent Steering
Zhanshuo Dong | Zhenyu Wen | Xiong Li | Jie Su | Jianbin Qin | Ye Yuan | Zhen Hong
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Zhanshuo Dong | Zhenyu Wen | Xiong Li | Jie Su | Jianbin Qin | Ye Yuan | Zhen Hong
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Tree of Thoughts (ToT) reasoning is fundamentally bottlenecked by its generate-then-score paradigm, which decodes every candidate branch into text and evaluates it through a decoupled scoring stage. Prior efficiency efforts primarily shrink the search breadth, leaving this prohibitive per-step textual evaluation intact. To resolve this, we propose ITOT (Implicit Tree of Thoughts), a training-free framework that recasts tree-structured reasoning as a continuous search within a model’s latent space. In this framework, branch quality is assessed intrinsically from hidden states via alignment with a contrastive prototype distilled from successful and failed traces. This elegantly collapses explicit branch enumeration and textual scoring into a single latent generate-evaluate-decide loop. Across six reasoning benchmarks and five LLM backbones, ITOT consistently improves mean accuracy over both the strongest tree-search and single-path baselines by roughly ten absolute points. Crucially, it achieves this while cutting token consumption by nearly an order of magnitude relative to explicit tree-search methods, operating even below the token budget of vanilla Chain of Thought. We will release our code to facilitate further research.