Zhanshuo Dong

Author directory

2026

Tree of Thoughts (ToT) reasoning is fundamentally bottlenecked by its generate-then-score paradigm, which decodes every candidate branch into text and evaluates it through a decoupled scoring stage. Prior efficiency efforts primarily shrink the search breadth, leaving this prohibitive per-step textual evaluation intact. To resolve this, we propose ITOT (Implicit Tree of Thoughts), a training-free framework that recasts tree-structured reasoning as a continuous search within a model’s latent space. In this framework, branch quality is assessed intrinsically from hidden states via alignment with a contrastive prototype distilled from successful and failed traces. This elegantly collapses explicit branch enumeration and textual scoring into a single latent generate-evaluate-decide loop. Across six reasoning benchmarks and five LLM backbones, ITOT consistently improves mean accuracy over both the strongest tree-search and single-path baselines by roughly ten absolute points. Crucially, it achieves this while cutting token consumption by nearly an order of magnitude relative to explicit tree-search methods, operating even below the token budget of vanilla Chain of Thought. We will release our code to facilitate further research.