We audit whether outperforming random activation directions establishes improved agent per- formance. Using Llama-3.1-8B in a molecular identification task, we compute every legal action’s exact probability of eventual success un- der optimal continuation. An entropy-weighted steering direction outperforms all 39 layer- and norm-matched random controls on 42 held-out molecular pools in a frozen comparison. Posthoc audits reveal the limits of this result. An untuned policy that chooses uniformly among legal actions also outperforms every random control, with no detectable difference from the steered policy; uncertainty intervals still admit modest effects in either direction. The steering direction ranks 10th among 36 directions when controls are matched to its construction, and its original-bank rank falls to 3/40 when pools sharing chemical scaffolds are grouped and weighted equally. Executed episodes show no detectable improvement over the unsteered baseline, with both policies performing near random under constrained decoding. These findings concern one model and task; they do not establish that steering cannot improve a competent agent. They demonstrate that su- periority to a random-direction bank does not by itself establish policy improvement. We propose five reporting controls covering trivial baselines, direction construction, inferential units, state selection and horizon, and executed behavior.

Text-to-SQL systems that use an LLM verifier typically calibrate its execute-or-defer threshold on English requests. We ask whether that threshold still makes the same decisions when the user writes in another language. Our evaluation is paired: the candidate SQL, the schema, the database, and the execution-derived correctness label are held fixed, and only the question changes, across seven languages and two open-weight verifiers. The verifiers rank candidates almost as well in every language, yet 14% to 27% of identical SQL receives a different execute-or-defer decision. The usual aggregates hide this. Selective risk stays near its English value because promotions and demotions offset, and coverage flags only the languages where the shift is one-sided. Underneath, both harms occur: correct queries are needlessly deferred, and incorrect queries that English withheld are executed; under Vietnamese, Llama executes 16.9% of all incorrect candidates, 22.9% of those it had deferred in English. Score-level corrections that equalize coverage barely change which queries run. Translating the request into English before verification helps more, lowering the flip rate and unsafe promotions on both verifiers, though it does not restore the English policy. Audits of multilingual verifiers should therefore report decision transitions, not only aggregate coverage and risk.

Sycophancy, a model abandoning a correct position under social pressure, is worst in languages with thin instruction-tuning coverage, yet interventions are built almost entirely in English. We extract an anti-sycophancy steering direction from English data alone and apply it unchanged across 38 languages of a parallel benchmark, on two model families. Mean transfer falls monotonically across resource tiers: the vector keeps 48 percent of its English effect in high-resource languages, 19 percent in low-resource, and 3 percent in zeroshot ones. Baseline sycophancy is worst in that bottom tier, so the intervention is least available where the problem is largest. Neither redosing to a language’s own activation norms nor re-extracting the direction from its own pairs closes the gap, and the strength the benchmark criterion would have chosen steepens it. A specificity probe shows the direction does not isolate factual resistance: at its frozen setting, factual correction falls from 72.3 to 52.0 percent on the full English probe. A second direction, built from a contrast that flatly corrects the user, improves factual correction instead and does survive translation, in four of five translated high-resource languages and in Bengali once a general shift toward disagreement is subtracted. The steepest failure belongs to the benchmark’s operationalisation rather than to factual-resistance steering in general. A reduced check at 70B finds the same gradient.

We ask whether giving different evidence to different LLM forecasters creates useful diversity when the total article-exposure count is held fixed. Across 708 resolved prediction-market contracts, we compare six-member ensembles whose forecasters either share the same two article pair or are distributed evenly across three pairs, with both conditions using twelve article exposures. On 609 pre-market-end contracts, evidence routing reliably increases within-ensemble dispersion for GPT-4.1-mini, Qwen3-30B, and OLMo-3.1-32B. An exact Brier-score decomposition separates changes in member error, dispersion, and ensemble error, while a grouped-variance analysis attributes nearly all of the dispersion increase to differences between assigned article pairs rather than repeated sampling. Greater dispersion, however, does not automatically improve the en- semble: only Qwen’s reduction in ensemble Brier survives multiplicity correction, while GPT and OLMo show no statistically robust net improvement. Post-hoc controls show that more balanced routing further increases dispersion at a fixed evidence union, while an alterna- tive Qwen readout changes channel attribution without establishing a different net effect. Evidence routing therefore consistently increases forecast diversity in this setting, but its value depends on whether that diversity outweighs any loss in individual forecast quality.

An existing product may meet an unmet need before the market recognises the connection. Testing whether a language model can identify such opportunities is difficult when historical outcomes may already be in its training data. We introduce LatentDemandBench, a contamination-resistant benchmark that generates fictional economies with known realisation probabilities and noisy documentary evidence. Solvers rank product–need pairings by their chance of realisation within a fixed horizon. Market commentary is generated separately from realisation, allowing us to evaluate ranking both among discussed pairings and where market attention supplies no ordering. On 30 matched economies, two frontier models beat a mention-count control but trail a supervised logistic ranker using fourteen features recovered from the same dossiers by a template-aware parser. The stronger model, GPT-5.6-TERRA, scores 0.669 in top-K-imputed AUC, with a paired reference-minus-model gap of +0.047 [+0.025, +0.074]. Aggregating three completions under the same final top-K budget yields 0.659, leaving a reference gap of +0.058 [+0.031, +0.095]. The deficit is not uniform: an exploratory analysis finds it smaller on unmentioned pairings than on those already discussed. We release the generator, harness and artifacts to support evaluation on fresh economies