Evaluations of sequential large language model (LLM) agents often report local action validity or final success. Delayed constructive tasks additionally require recoverability, meaning each accepted edit must leave the partial artifact completable within the remaining budget. We introduce a measurement protocol across Word Ladder, an alloy composition proxy, and the fitness landscape of the B1 domain of protein G (GB1). We call an episode surface clean when every action is well formed and locally admissible. Across six models, Word Ladder exposes interface failure with 36.1% local validity. Alloy and GB1 episodes are often surface clean, yet only 16.0% and 6.8% of them succeed. Exact enumeration of 625 GB1 variants over budgets from zero to six decomposes failure. Among surface clean episodes, 51.4% cross a recoverability boundary, while 14.1% of the remaining episodes succeed. Bounded composition audits and intervention checks connect bottlenecks to repair classes. These results establish recoverability as a decomposable and auditable evaluation axis that informs interventions for sequential agents.

Aligned language models are evaluated for whether they state falsehoods and whether users prefer their answers. Neither axis asks whether an answer carries what a reader needs to decide. We name that failure mode Pragmatic Reticence and build an instrument to measure it. A generator answers a passage-and-question prompt in free form, never seeing the answer options, and a frozen reader, never seeing the passage, decides from that answer alone. A natural language inference judge scores whether the answer conveys the passage’s decision-critical sentence. Across 47 generators the median non-reasoning model reaches a critical-fact recall of 0.28, while every reasoning-tuned checkpoint exceeds 0.74 on the whole response. Supervised fine-tuning and preference optimization do not close that gap. For models that emit a reasoning trace, the recalled fact usually sits inside the trace rather than in the answer that follows it, so a deployment that hides the trace hides the recall. In those models a trace-free prompt restores it, so the omission is a format effect. An answer can be true, preferred, and still not enough to act on.

When an aligned language model declines a request, it almost always states a reason. Those reasons are read as evidence about the refusal, yet whether they name what actually triggered it has not been measured. We introduce a counterfactual audit of the cited reason, removing the reason-bearing feature to test necessity and inserting it into a benign prompt to test sufficiency. The two tests partition cited reasons into four kinds: faithful, over-cited, under-specified, and confabulated. We audit 22 models from nine developers on 840 prompts pooled from seven public safety benchmarks. Across the audited refusals, 57.2% cite a reason that fails at least one test, though faithful remains the largest category for 21 of 22 models. Over-citation is highest in the models that refuse most broadly. Paired sweeps over two families’ post-training checkpoints place its rise at supervised fine-tuning, not at later preference optimization. Probes on open-weight models tie these labels to internal representations. Category rates hold under a different-family judge, and the faithful-versus-over-cited ordering holds under an own-reason audit. We release the framework, the counterfactual variants, and per-model labels.

Self-consistency treats agreement among sampled chains of thought as evidence that the answer is correct. We show that this reading fails predictably. Across 5 reasoning datasets and 5 generators, consistent-but-wrong (CBW) rates track the task rather than the generator, and the pattern holds across agreement thresholds and sample budgets. On StrategyQA, at least 15% of answers are wrong at 90% self-agreement. We prove that no procedure reading only the answer distribution can always separate a question whose majority is right from one whose majority is wrong. We propose Risk-Controlled Self-Consistency (RISC), which combines the majority ratio with a verifier signal and calibrates an acceptance threshold by Learn-then-Test. It certifies the risk target in domain at finite sample size, and accepts nothing when no threshold qualifies. Against self-consistency alone, pooled selective error falls 3.01pp at 0.50 coverage; per-task gains are mixed. At 0.9 coverage, a local generator with a lightweight verifier costs ~9× less than an all-API baseline and shows no detectable selective-error gap on four of five datasets. It trails on MATH500. Higher agreement is not always higher accuracy. Calibration narrows the gap, but more sampling usually does not.