Daniel Hacker
Author directory2026
Saying vs. Seeing: Evaluating LLM Conflict Awareness and Resolution Preference under Latent Claim-Evidence Disagreement
Daniel Hacker | Kfir Bar
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Daniel Hacker | Kfir Bar
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Language models are routinely exposed to contexts where an authoritative claim conflicts with a conclusion derivable from underlying evidence. Unlike explicit contradictions, such conflicts are latent: they emerge only after a full multi-step derivation, a setting current benchmarks largely overlook. We introduce and publicly release CLECK (CLaim Evidence Conflict of Knowledge), a counterfactual benchmark of 5,922 instances (evaluated on a balanced 2,669-instance subset) probing this blind spot across eleven reasoning strategies. Among instruction-tuned models, the capability to recover both competing answers does not translate into recognizing their conflict at inference: models default to the authoritative claim. When the conflict does surface in a model’s reasoning, preference shifts by 34 percentage points toward the derived evidence on average. Reasoning strategies and evidence load further modulate this behavior, and while chain-of-thought and claim hedging both improve awareness, they shift resolution in opposite directions: chain-of-thought toward the claim, hedging toward the evidence.