Austin Long
Author directory2026
Are Latent Reasoning Tokens Unnecessary for Model Performance?
Austin Long | Colin Paterson | Jing Xu
Proceedings of the 9th BlackboxNLP Workshop: Analyzing and Interpreting Neural Networks for NLP
Austin Long | Colin Paterson | Jing Xu
Proceedings of the 9th BlackboxNLP Workshop: Analyzing and Interpreting Neural Networks for NLP
Training large language models to perform latent reasoning may improve performance, but the lack of legible reasoning poses challenges for model interpretability and safety assurance. This challenge has motivated interpretability research that aims to characterise the functional role of latent reasoning tokens. Some studies indicate that latent reasoning tokens are effective, whereas other work suggests that these tokens are under-utilised. In this paper, we consider the tension between existing findings by reproducing prior work which shows that some latent reasoning tokens are not necessary. Two interesting observations emerge from our experiments. First, we find that latent reasoning models often generate redundant tokens due to the fixed token budgets. Second, we observe that explicit reasoning models fine-tuned on gold reasoning traces may copy the result of the most recent reasoning step when generating an answer.