2026

Aligned language models are evaluated for whether they state falsehoods and whether users prefer their answers. Neither axis asks whether an answer carries what a reader needs to decide. We name that failure mode Pragmatic Reticence and build an instrument to measure it. A generator answers a passage-and-question prompt in free form, never seeing the answer options, and a frozen reader, never seeing the passage, decides from that answer alone. A natural language inference judge scores whether the answer conveys the passage’s decision-critical sentence. Across 47 generators the median non-reasoning model reaches a critical-fact recall of 0.28, while every reasoning-tuned checkpoint exceeds 0.74 on the whole response. Supervised fine-tuning and preference optimization do not close that gap. For models that emit a reasoning trace, the recalled fact usually sits inside the trace rather than in the answer that follows it, so a deployment that hides the trace hides the recall. In those models a trace-free prompt restores it, so the omission is a format effect. An answer can be true, preferred, and still not enough to act on.
When an aligned language model declines a request, it almost always states a reason. Those reasons are read as evidence about the refusal, yet whether they name what actually triggered it has not been measured. We introduce a counterfactual audit of the cited reason, removing the reason-bearing feature to test necessity and inserting it into a benign prompt to test sufficiency. The two tests partition cited reasons into four kinds: faithful, over-cited, under-specified, and confabulated. We audit 22 models from nine developers on 840 prompts pooled from seven public safety benchmarks. Across the audited refusals, 57.2% cite a reason that fails at least one test, though faithful remains the largest category for 21 of 22 models. Over-citation is highest in the models that refuse most broadly. Paired sweeps over two families’ post-training checkpoints place its rise at supervised fine-tuning, not at later preference optimization. Probes on open-weight models tie these labels to internal representations. Category rates hold under a different-family judge, and the faithful-versus-over-cited ordering holds under an own-reason audit. We release the framework, the counterfactual variants, and per-model labels.
Self-consistency treats agreement among sampled chains of thought as evidence that the answer is correct. We show that this reading fails predictably. Across 5 reasoning datasets and 5 generators, consistent-but-wrong (CBW) rates track the task rather than the generator, and the pattern holds across agreement thresholds and sample budgets. On StrategyQA, at least 15% of answers are wrong at 90% self-agreement. We prove that no procedure reading only the answer distribution can always separate a question whose majority is right from one whose majority is wrong. We propose Risk-Controlled Self-Consistency (RISC), which combines the majority ratio with a verifier signal and calibrates an acceptance threshold by Learn-then-Test. It certifies the risk target in domain at finite sample size, and accepts nothing when no threshold qualifies. Against self-consistency alone, pooled selective error falls 3.01pp at 0.50 coverage; per-task gains are mixed. At 0.9 coverage, a local generator with a lightweight verifier costs ~9× less than an all-API baseline and shows no detectable selective-error gap on four of five datasets. It trails on MATH500. Higher agreement is not always higher accuracy. Calibration narrows the gap, but more sampling usually does not.