Daniel Hacker

Author directory

2026

Language models are routinely exposed to contexts where an authoritative claim conflicts with a conclusion derivable from underlying evidence. Unlike explicit contradictions, such conflicts are latent: they emerge only after a full multi-step derivation, a setting current benchmarks largely overlook. We introduce and publicly release CLECK (CLaim Evidence Conflict of Knowledge), a counterfactual benchmark of 5,922 instances (evaluated on a balanced 2,669-instance subset) probing this blind spot across eleven reasoning strategies. Among instruction-tuned models, the capability to recover both competing answers does not translate into recognizing their conflict at inference: models default to the authoritative claim. When the conflict does surface in a model’s reasoning, preference shifts by 34 percentage points toward the derived evidence on average. Reasoning strategies and evidence load further modulate this behavior, and while chain-of-thought and claim hedging both improve awareness, they shift resolution in opposite directions: chain-of-thought toward the claim, hedging toward the evidence.