Ziyi Ding
Author directory2026
Causal Tongue-Tie: LLMs Can Encode Causal Direction, But Their Yes/No Outputs Fail to Express
Ziyi Ding | Xiao-Ping Zhang
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Ziyi Ding | Xiao-Ping Zhang
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
We find a mismatch between the causal direction recoverable from a large language model’s hidden state and its final Yes/No answer. On anti-commonsense CLadder (Jin et al., 2024) items, a frozen linear probe recovers the evidence-supported direction with accuracy 0.97–1.00, while ordinary Yes/No accuracy remains 0.35–0.53. This effect disappears in randomly initialized models, so it depends on the trained representation rather than probe capacity alone. Erasing the recovered direction reduces the model’s internal comparison of paired prompts by 96%, and replacing the full hidden state flips 57% of previously wrong answers. A 2 × 2 interface control further shows that changing only the answer tokens does not fix the error; explicitly contrasting the two candidate directions raises accuracy to 0.95–1.00 across Qwen, Mistral, and Llama. We call this mismatch Causal Tongue-Tie. It does not establish human-like causal understanding. Our main finding is that a wrong Yes/No answer can coexist with an evidence-aligned direction that the tested probe recovers from the same forward pass. Output-only accuracy therefore cannot distinguish an absent internal signal from a present but unexpressed one.