Chain-of-thought (CoT) prompting improves performance on complex reasoning tasks, but often triggers unnecessary computation on simpler queries. A natural approach is to use the model’s own confidence as a training-free signal to gate this decision, answering directly when confident and invoking CoT only when uncertain. Across four training-free confidence signals, three model families, and seven reasoning benchmarks, we find that the effectiveness of confidence-gated CoT varies, and it is unclear how to predict whether it will work for a new model. We address this with two key contributions. First, we identify that gating fails in the following circumstances: signal saturation, where confidence scores collapse to a narrow range and lose discriminative power; and selection bias, when confidence-gated routing sends the longest queries to CoT. Second, we propose a novel Pareto-optimal threshold selection method. This has the dual ability to preserve accuracy while reducing costs where the confidence signal is strong, and maintains accuracy where it is weak. Where the confidence signal is strong, gating reduces CoT usage by 30% without sacrificing accuracy. These findings provide practical guidance for evaluating and deploying models that selectively use CoT.

Recent work has demonstrated that using chain of thought (CoT), on soft-reasoning problems such as analytical and commonsense reasoning, often yields limited or even negative performance gains. CoT can also be unfaithful to the model’s actual reasoning. This paper investigates dynamics and unfaithfulness of CoT in soft-reasoning tasks across instruction-tuned, reasoning and reasoning-distilled models. Our findings show that distilled‐reasoning models rely heavily on CoT for these tasks, while instruction‐tuned and reasoning models often use it post‐hoc. Additionally, we find that CoT can steer model predictions without faithfully reflecting reasoning, indicating a disconnect between CoT influence and faithfulness.