Span-level hallucination metrics aggregate outputs whose references contain hallucinated spans with outputs whose references are empty. Under scoring rules that award a perfect match to an empty prediction against empty gold, the aggregate entangles detection performance with empty-gold prevalence. We write the exact decomposition as s=ea+(1-e)d, where e is empty-gold prevalence, a is performance on empty-gold items, and d is performance on non-empty-gold items. Under the benchmark’s metric conventions, an empty submission identifies e from leaderboard values without access to closed labels. Applied to a multilingual vision-language hallucination benchmark, this recovers rates from 0.1549 to 0.2554. Assuming perfect empty-gold performance gives a maximum contribution equal to 47.3–64.8% of the mean team score and a conservative lower bound on d. Under this bound, the highest-ranked language moves from best for 17 of 18 teams to best for none and falls from rank 1 to rank 3. Released annotations show that union-over-annotators gold retains 4.09 times as many characters as majority-agreement gold, while 71.7% of annotated spans were marked by one of three annotators. Cross-split comparisons therefore require reporting both empty-gold prevalence and system performance on empty-gold items alongside aggregate scores.

Running a tool agent n times and keeping the best run is worth 2.19–22.37 points on τ²-bench, a floor any scaffold spending n times the calls has to clear. Choosing among those runs, for no additional model calls, is worth anywhere from −23.68 to +12.28 depending on which rule does the choosing. We evaluate 9 selectors that read only the agent’s own trajectory across 28 model × domain cells and 10 403 released trials, and add an LLM judge that reads the trajectories but never a reward and never the evaluation criteria. Every arm is therefore reward-free: the nine rules are verifier-free as well, the judge is not, and neither sees a label. Against a mean best-of-K oracle band of 12.29 points, which is what a selector holding the reward would collect, the judge recovers 32.7% and the best free rule 21.0%, leaving 67.3% of that band unreachable without the label. One measurable property of the rollouts orders the task distributions, the no-mode rate, the share of tasks on which no two of the K rollouts emit the same write-action sequence, and what it predicts is when agreement is exploitable rather than when selection is possible. Where agreement-based rules are nearly inert the judge is worth +5.95 against +0.53; where the rollouts are undifferentiated it returns 1.19 against a band of 14.4 points; on a third domain it returns −0.64 and loses to a free heuristic costing one fewer call.

Inference-time methods buy accuracy with extra compute: sample several reasoning traces and combine them. They are compared at a matched number of samples, and that protocol hides two things. Samples do not cost equal tokens, so matched samples is not matched budget; and it never asks whether a cheaper source of diversity already reaches the same ceiling. We measure both on cross-lingual aggregation, which votes across traces generated in different languages, using 32,992 traces from Qwen38B across 11 languages on two item-parallel corpora. At a matched trace budget on MGSM, one trace from each of the eight strongest languages buys no ceiling and no vote that resampling English eight times does not: the per-item oracle is 0.992 for samples against 0.989 for languages, the vote 0.966 against 0.956, and the language ballot costs 1.09× the tokens. Over the full pool the eighty traces from the other ten languages solve 0 of 250 items that English’s own eight do not. On MMLU-ProX-Lite, far from saturation, the languages do hold coverage English never reaches (0.895 against 0.685 over the full 88-trace pool against English’s eight, 26 items) — and at a matched budget of eight traces the language ballot there is still the weaker vote (0.453 against 0.487). What blocks the conversion is not error correlation, which is low: an independence model built from the measured marginals over-predicts the vote by 5.4 and 48.0 points, because the independence sits on items every language already answers and vanishes on the items that decide the score, where wrong answers concentrate on a single value (0.464 of erring languages). Three selectors computable from the released traces without gold access all fail to beat the best single language. We release both pools with per-trace token counts so any aggregation or routing rule can be scored against these ceilings by re-scoring rather than generating.

Comparisons of retrieved and randomly selected exemplars in mathematical in-context learning confound query relevance with changes in the demonstration pool. We separate these effects by assigning each query the demonstrations retrieved for another query. This derangement preserves the aggregate demonstrations, counts, token budget and retrieval-induced selection bias while changing the query-to-exemplar assignment.

On GSM8K, derangement changes accuracy by −0.24 pp (90% CI [−0.85, +0.37]). The pooled effect is equivalent to zero within 1.0 pp across conditions that include embedding retrieval, a gold-computational-graph oracle and a 7B model. On MathQA, the change is −8.36 pp. Near-duplicates occur above cosine 0.95 for 64.6% of MathQA queries and no GSM8K queries. Removing them changes the matching term by −13.56 pp relative to queries unaffected by removal.

Pairing each retrieved problem with another retrieved rationale changes accuracy by −22.40 to −52.24 pp. Across 8 conditions and 3 model families, this change correlates with excess copying of demonstration answers (Spearman −0.976). Across 7 models, the correlation between the accuracy change and unpermuted accuracy is +0.286. These results support reporting pool near-duplicate density alongside retrieval gains and checking problem-rationale alignment during prompt construction, since retrieval metrics remain unchanged under this within-demonstration permutation.

DocVQA and InfographicVQA are scored with average normalized Levenshtein similarity (ANLS), and a DocVQA number is read as a measure of how well a model reads a page, but the lmms-eval and VLMEvalKit harnesses that compute it append a one-line terse-answer suffix to every question. ANLS divides edit distance by the length of the longer string, so a correct answer inside a sentence more than twice its length falls below the 0.5 threshold and scores zero, and removing only the suffix moved Qwen2.5-VL-3B’s DocVQA ANLS from 94.2 to 7.5. On that model ANLS given a short answer (within two tokens of the longest gold) stayed between 93.9 and 95.7 across three suffixes whose scores spanned 94.2 to 65.3, so the reported score followed the share of short answers more than the share that contained the gold. Two suffixes that lmms-eval defines for the same task differed by 28.8 points, and under one of them 79% of the answers ANLS scored zero contained the gold answer, all 305 of which a judge with negated and two-candidate controls accepted as the committed answer. On a 317-question subset, four hosted models lost between 34.4 and 89.5 points without the suffix (Gemini 3.8 Flash fell from 92.0 to 2.5), and with the suffix in place, deleting Markdown emphasis markers raised Claude Opus 5.5 by 10.9 points. On Qwen2.5-VL-3B a length-gated span scorer moved 13.3 points across the five suffixes where ANLS moved 86.6, and it scored a submission of the page’s whole OCR at 1.3, where best-subsequence exact match and the learned PEDANTS classifier gave 93.0 and 98.6. On ChartQA the suffix was worth 28.4 points of relaxed accuracy to Qwen2.5-VL-3B. We release the scorer and recommend reporting the prompt suffix and output post-processing with every document-VQA score.