Frontier reasoning systems generate chain-of-thought (CoT) traces meant to reveal how they reach answers, yet these traces systematically misrepresent the underlying computation. A tempting explanation is bandwidth: each token carries few bits while the residual stream encodes many, so sufficiently complex tasks would make faithful verbalization impossible in principle. We formalize this intuition and locate its limits. A capacity bound rules out faithful CoT whenever the information the model uses exceeds the trace’s token capacity; this condition requires the answer to be longer than the trace, so standard CoT settings do not reach it. A per-step information-loss conjecture is empirically falsified on six of seven benchmarks. A constructive result shows faithful traces remain achievable for polynomial-size circuit families, and an information-bottleneck experiment supports a faithfulness–accuracy tradeoff. Measuring critic-score gaps on seven benchmarks for three open-weight models, complemented by perturbation sensitivity, we find no benchmark provably in the above-capacity regime. The capacity bound is architecture-independent, but at evaluated trace lengths it does not bind: observed unfaithfulness is attributable to training rather than to the token interface, and is therefore potentially correctable.

A retrieval-augmented language model returns the correct free-cash-flow figure from Microsoft’s 10-K filing. Replace the filing with a topical distractor containing no figure, and the model returns the same number: the document was never needed. We introduce RAG-CF, a training-free black-box probe that issues a per-query verdict on whether an apparently correct RAG answer is source-independent, meaning produced without the retrieved documents. The probe perturbs document context under four counterfactual operators, regenerates the answer, and converts the aggregated mismatch into a split-conformal P-value with a finite-sample Type-I bound. Across 60 configurations spanning five benchmarks, four generators, and three retrievers, the FPR-adjusted flagged rate, (raw − α)/(1 − α), ranges from 13% to 56% (raw 21% to 60%); the instrument’s FPR is domain-dependent. A held-out no-retrieval label set confirms: 78% of flagged answers remain correct without documents, against 22% of unflagged; the gap is preserved under LLM-judge equivalence. Source-independence does not decrease with retriever quality. Accuracy alone conflates grounded and parametric success; reporting the source-independence rate alongside it would make the distinction visible on every future RAG benchmark.

The EU Artificial Intelligence Act now requires providers to mark generated text so its origin can be checked. In regulated industries the duty is sharper: a marker must never flag third-party writing, survive routine review edits, and work on outputs often a sentence long. We call these the regulated-provenance triad and prove that no bounded-distortion watermark perturbing a model’s token distribution can satisfy all three below a critical length, the provenance horizon. The proof combines the Le Cam two-point method with a strong data-processing inequality for the edit channel; a matching achievability bound for the green-list family under a memoryless resampling channel makes the scaling order-tight. The barrier rises at negligible-distortion budgets, not at deployed ones. The contraction factor entering the bound is a proven upper bound for the memoryless resampling channel and a calibrated estimate for realistic paraphrase, under which we estimate most sentence-length regulated outputs fall below the horizon. Relocating provenance to a tamper-evident log read through a conformally calibrated retrieval test sidesteps the barrier: under exchangeable calibration its false-positive rate is at or below the target at every length. Coverage carries no guarantee and degrades under an adaptive editor.

LLM-as-judge leaderboards now decide what gets trained, deployed, and funded, yet years of bias patching have not asked whether the goals those patches pursue are jointly achievable. We show they are not. Casting LLM-as-judge evaluation as a social welfare functional over score profiles indexed by judge, criterion, and alternative, we prove that no aggregation rule can simultaneously satisfy four fairness axioms (Universal Domain, Weak Pareto, Independence of Irrelevant Alternatives [IIA], and Judge-Anonymity) in the ordinal regime and, among scale-invariant aggregators, in the cardinal regime. A fifth axiom, Position-Invariance, captures upstream pipeline bias and is securable independently. The impossibility admits a constructive counterpart: a four-face achievability frontier with one witness per dropped axiom. The Borda-Conformal Hybrid (BCH) occupies the dropped-IIA face, satisfies the other four axioms, and admits a structural conformal bound relating its IIA-violation probability to the calibration-set gap distribution. A 200-prompt audit on MT-Bench, AlpacaEval 2, and Arena-Hard finds IIA-violation rates from 18.5% to 38.0% across the seven aggregators with measurable IIA; a 50-prompt human-jury comparator confirms a non-zero human IIA-violation baseline consistent with known ranking pathologies in the choice-experiment literature. An audit toolkit and reporting protocol accompany the paper.

Conformal prediction supplies finite-sample coverage guarantees for individual language-model responses. We show that per-step conformal calibration need not preserve this guarantee along a multi-step agent trajectory: at a nominal miscoverage level of 10%, per-step split conformal delivers 83.5% trajectory score-threshold miscoverage on a fifteen-step web-navigation agent, and we prove lower bounds in three regimes of agent dynamics: i.i.d. resampling, memoryless covariate shift, and state-dependent Markov transitions. The i.i.d. regime alone accounts for the bulk of the empirical gap; the third regime, novel to this work, gives, to our knowledge, the first separation of the trajectory union event from i.i.d. resampling under marginal-stationary, non-trivial Markov dynamics, a setting in which prior marginal-drift analyses are silent. The separation persists under exact score stationarity and, for non-degenerate score laws, holds at any calibration sample size up to an O(T/sqrt(n)) finite-sample slack. A polynomial alpha-spending schedule restores end-to-end coverage to within 1.1 points of nominal across five agentic-NLP benchmarks and, by direct substitution, lifts process-reward-model step verifiers – which on their own miscover trajectories at 31.5-62.5% – to 9.6-10.6%, near the nominal target.

Many natural language processing (NLP) tasks rely on labeled data to train machine learning models with high performance. However, data annotation is time-consuming and expensive, especially when the task involves a large amount of data or requires specialized domains. Recently, GPT-3.5 series models have demonstrated remarkable few-shot and zero-shot ability across various NLP tasks. In this paper, we first claim that large language models (LLMs), such as GPT-3.5, can serve as an excellent crowdsourced annotator when provided with sufficient guidance and demonstrated examples. Accordingly, we propose AnnoLLM, an annotation system powered by LLMs, which adopts a two-step approach, explain-then-annotate. Concretely, we first prompt LLMs to provide explanations for why the specific ground truth answer/label was assigned for a given example. Then, we construct the few-shot chain-of-thought prompt with the self-generated explanation and employ it to annotate the unlabeled data with LLMs. Our experiment results on three tasks, including user input and keyword relevance assessment, BoolQ, and WiC, demonstrate that AnnoLLM surpasses or performs on par with crowdsourced annotators. Furthermore, we build the first conversation-based information retrieval dataset employing AnnoLLM. This dataset is designed to facilitate the development of retrieval models capable of retrieving pertinent documents for conversational text. Human evaluation has validated the dataset’s high quality.

Commonsense generation aims to generate a realistic sentence describing a daily scene under the given concepts, which is very challenging, since it requires models to have relational reasoning and compositional generalization capabilities. Previous work focuses on retrieving prototype sentences for the provided concepts to assist generation. They first use a sparse retriever to retrieve candidate sentences, then re-rank the candidates with a ranker. However, the candidates returned by their ranker may not be the most relevant sentences, since the ranker treats all candidates equally without considering their relevance to the reference sentences of the given concepts. Another problem is that re-ranking is very expensive, but only using retrievers will seriously degrade the performance of their generation models. To solve these problems, we propose the metric distillation rule to distill knowledge from the metric (e.g., BLEU) to the ranker. We further transfer the critical knowledge summarized by the distilled ranker to the retriever. In this way, the relevance scores of candidate sentences predicted by the ranker and retriever will be more consistent with their quality measured by the metric. Experimental results on the CommonGen benchmark verify the effectiveness of our proposed method: (1) Our generation model with the distilled ranker achieves a new state-of-the-art result. (2) Our generation model with the distilled retriever even surpasses the previous SOTA.

Example sentences for targeted words in a dictionary play an important role to help readers understand the usage of words. Traditionally, example sentences in a dictionary are usually created by linguistics experts, which are labor-intensive and knowledge-intensive. In this paper, we introduce the problem of dictionary example sentence generation, aiming to automatically generate dictionary example sentences for targeted words according to the corresponding definitions. This task is challenging especially for polysemous words, because the generated sentences need to reflect different usages and meanings of these targeted words. Targeted readers may also have different backgrounds and educational levels. It is essential to generate example sentences that can be understandable for different backgrounds and levels of audiences. To solve these problems, we propose a controllable target-word-aware model for this task. Our proposed model can generate reasonable examples for targeted words, even for polysemous words. In addition, our model allows users to provide explicit control over attributes related to readability, such as length and lexical complexity, thus generating suitable examples for targeted audiences. Automatic and human evaluations on the Oxford dictionary dataset show that our model can generate suitable examples for targeted words with specific definitions while meeting the desired readability.