Contrastive Neuron Regulation for Chain-of-Thought Reasoning

Zitao Su, Xinyu Tang, Xin Zhao


Abstract
Chain-of-thought (CoT) reasoning is commonly improved through external mechanisms such as prompting, sampling, and decoding, while the internal units shaping reasoning trajectories remain less understood. We study CoT from the perspective of reasoning trajectory structure and find that incorrect trajectories are more redundant and repetitive, whereas correct ones are more lexically diverse and information-dense. Motivated by this observation, we propose Contrastive Neuron Regulation (CNR), a sparse, backbone-frozen framework for neuron-level regulation of CoT reasoning. CNR constructs correctness-controlled expanded–concise pairs of correct trajectories, identifies MLP neurons associated with the expansion contrast, and sparsely suppresses them during generation. We consider both a fixed-coefficient inference-time intervention and an adaptive variant that learns neuron-specific suppression coefficients on a subject-specific development split while keeping the backbone frozen. Across two reasoning models and seven MATH subjects, CNR improves macro-average accuracy over the no-intervention baseline by 2.6 and 3.2 percentage points, respectively, while intervening on less than 1% of MLP neurons. The code is available at https://github.com/RUCAIBox/Contrastive-Neuron-Regulation.
Anthology ID:
2026.emnlp-main.1846
Volume:
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Month:
October
Year:
2026
Address:
Budapest, Hungary
Editors:
Hung-Yi Lee, Sunipa Dev, Jessy Li, Desmond Elliott
Venue:
EMNLP
SIG:
SIGDAT
Publisher:
Association for Computational Linguistics
Note:
Pages:
41986–42005
Language:
URL:
https://aclanthology.org/2026.emnlp-main.1846/
DOI:
Bibkey:
Cite (ACL):
Zitao Su, Xinyu Tang, and Xin Zhao. 2026. Contrastive Neuron Regulation for Chain-of-Thought Reasoning. In Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing, pages 41986–42005, Budapest, Hungary. Association for Computational Linguistics.
Cite (Informal):
Contrastive Neuron Regulation for Chain-of-Thought Reasoning (Su et al., EMNLP 2026)
Copy Citation:
PDF:
https://aclanthology.org/2026.emnlp-main.1846.pdf