Critical or Compliant? The Double-Edged Sword of Reasoning in Chain-of-Thought Explanations

Eunkyu Park, Wesley Hanwen Deng, Vasudha Varadarajan, Mingxi Yan, Gunhee Kim, Maarten Sap, Motahhare Eslami


Abstract
Explanations are often promoted as tools for transparency, but they can also foster blind trust; users may assume reasoning is correct whenever model outputs appear acceptable. We study this double-edged role of Chain-of-Thought (CoT) explanations in multimodal moral scenarios by systematically perturbing reasoning chains and manipulating delivery tones. Specifically, we perturb reasoning chains generated by vision-language models (VLMs) with three error types (omissions, contradictions, hallucinations) and measure their effects on user trust and error detection through a study with 800 participants, alongside a parallel detection task with models. Our findings reveal two key effects: (1) users often equate trust with outcome agreement regardless of flaws in the model’s reasoning, and (2) confident tones suppress scrutiny while leaving agreement and self-reported trust largely unchanged. These results highlight how CoT explanations can simultaneously clarify and mislead in tandem, and point to user-side scrutiny, not just model-side faithfulness, as a critical dimension for calibrated trust in AI reasoning. All code will be released publicly upon acceptance.
Anthology ID:
2026.emnlp-main.2050
Volume:
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Month:
October
Year:
2026
Address:
Budapest, Hungary
Editors:
Hung-Yi Lee, Sunipa Dev, Jessy Li, Desmond Elliott
Venue:
EMNLP
SIG:
SIGDAT
Publisher:
Association for Computational Linguistics
Note:
Pages:
46688–46708
Language:
URL:
https://aclanthology.org/2026.emnlp-main.2050/
DOI:
Bibkey:
Cite (ACL):
Eunkyu Park, Wesley Hanwen Deng, Vasudha Varadarajan, Mingxi Yan, Gunhee Kim, Maarten Sap, and Motahhare Eslami. 2026. Critical or Compliant? The Double-Edged Sword of Reasoning in Chain-of-Thought Explanations. In Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing, pages 46688–46708, Budapest, Hungary. Association for Computational Linguistics.
Cite (Informal):
Critical or Compliant? The Double-Edged Sword of Reasoning in Chain-of-Thought Explanations (Park et al., EMNLP 2026)
Copy Citation:
PDF:
https://aclanthology.org/2026.emnlp-main.2050.pdf