As retrieval-augmented generation (RAG) systems scale, ensuring faithful grounding in external evidence remains challenging. Large language models may still rely on parametric knowledge when it conflicts with retrieved information. We propose Grounded Decoding, a training-free decoding framework that improves factual consistency in RAG without modifying model parameters. At each generation step, our method constructs two matched-prompt distributions: a full RAG distribution conditioned on the query, retrieved documents, and generated prefix, and a retrieval-only distribution conditioned solely on the retrieved evidence and the same prefix. We derive the final next-token distribution as the unique solution to a KL-barycenter objective over the probability simplex, resulting in a normalized geometric fusion of the two distributions. The formulation recovers standard RAG when the grounding weight is zero and progressively shifts probability mass toward retrieved evidence as grounding strength increases. We further introduce a conflict-aware adaptive weighting scheme that adjusts grounding strength based on distributional disagreement and retriever confidence. Experiments on ALCE, Natural Questions, and FActScore show consistent improvements in factual accuracy and citation quality over standard RAG and competitive decoding-time baselines while maintaining fluency. These results suggest that probability-level fusion offers an effective and efficient alternative to logit-level intervention for faithful RAG decoding.

Large language models can produce correct final answers with reasoning traces that are incomplete, inconsistent, or weakly coupled to the answer. We study whether training can increase this coupling in structured reasoning domains. We introduce Counterfactual Sensitivity Regularization (CSR), a training objective that creates verifier-approved counterfactual edits to reasoning traces, such as changing an arithmetic operator or a logical relation, and penalizes models when the answer distribution remains unchanged after an edit that should change the computation. We evaluate this behavior with Counterfactual Outcome Sensitivity (COS), the fraction of originally correct examples on which a validity-breaking trace edit changes the final answer. Because COS is aligned with the CSR objective by construction, we report it jointly with counterfactual target accuracy, target coverage, and null-edit preservation, and alongside evidence collected outside the training loop: natural-unfaithfulness audits, external formal verification, input contrast sets, and trace-dependence probes. Across arithmetic, formal logic, multi-hop question answering, and biomedical QA, CSR substantially improves edit sensitivity while incurring small accuracy trade-offs. The gains are largest in domains where the edited operators can be identified and verified reliably; performance degrades as verifier precision falls. These results support CSR as a practical regularizer for trace-sensitive reasoning, not as a complete solution to explanation faithfulness.

Parameter-efficient fine-tuning (PEFT) adapts large pre-trained models by updating only a small subset of parameters. Recently, Representation Fine-Tuning (ReFT) has emerged as an effective alternative. ReFT shifts the fine-tuning paradigm from updating model weights to directly manipulating hidden representations that capture rich semantic information, and outperform state-of-the-art PEFTs in standalone settings. However, its application in Federated Learning (FL) remains challenging due to heterogeneity in clients’ data distributions, model capacities, and computational resources. To address these challenges, we introduce Federated Representation Fine-Tuning (FedReFT), a novel approach to fine-tune clients’ hidden representations. FedReFT applies sparse intervention layers to steer hidden representations directly, offering a lightweight and semantically rich fine-tuning alternative ideal for edge devices. However, representation-level updates are especially vulnerable to aggregation mismatch under different task heterogeneity, where naive averaging can corrupt semantic alignment. To mitigate this issue, we propose All-But-Me (ABM) aggregation, where each client receives the aggregated updates of others and partially incorporates them, enabling stable and personalized learning by balancing local focus with global knowledge. We further design an adaptive update strategy inspired by Test-Time Computing (TTC) to balance local and global contributions under heterogeneous conditions. FedReFT achieves state-of-the-art performance on commonsense reasoning, arithmetic reasoning, and GLUE benchmarks, while delivering 1x–49x higher parameter efficiency compared to leading LoRA-based methods.

Proxy optimization, where AI systems exploit evaluator weaknesses rather than improve intended objectives, threatens both reinforcement learning (reward hacking) and LLM alignment (evaluator gaming). We introduce the Evaluator Stress Test (EST), an invariance-based framework that detects proxy gaming by separating exploitable sensitivity (e.g., format, physics bugs) from content-driven improvements using controlled perturbations with semantic validity audits. We validate EST across both domains. In RL, across 15 environments and 5 algorithms (2,156 expert-annotated episodes), EST achieves 78.4% precision and 81.7% recall. In LLM alignment, across 4 tasks, 2 model scales, 2 training methods, and 2 judges (1,200 human-annotated instances), EST achieves 74.2% precision and 78.6% recall with early warning signals preceding quality decline. Cross-domain analysis reveals that proxy-true correlation tracking transfers directly between domains, while perturbation design requires domain adaptation. Closed-loop mitigation improves human win-rate by 8.3 points (LLM) and reduces hacking by 54.6% (RL). We release benchmarks for both domains: 2,156 RL episodes and 1,200 LLM instances.

Direct Preference Optimization removes the explicit reward model of RLHF but inherits its central failure mode: the preference-reward proxy can keep improving while downstream reasoning, factuality, or safety degrades. We ask whether this reward overoptimization, which we label *validation collapse* as an operational name for the trajectory we measure, is visible in weight-space geometry. Drawing on Random Matrix Theory and the Heavy-Tailed Self-Regularisation hypothesis, we propose Spectral DPO, which augments the DPO objective with a stochastic penalty computed from a truncated top-k Lanczos estimate of the spectra of effective adapted projections. The penalty holds the fitted tail exponent near a heavy-tailed reference, caps scale-normalised growth of the largest singular direction, and matches the estimated spectrum to a heavy-tailed target. Across ten matched seeds, three model families, and five benchmarks spanning instruction following, dialogue, mathematical reasoning, factuality, and adversarial robustness, Spectral DPO gains 2.33–2.42 points over the strongest included baseline and sustains accuracy through ten epochs where vanilla DPO degrades. Controls and a post-hoc spectral repair rule out norm clipping, KL anchoring, and length inflation, and support the relevance of spectral geometry without identifying a single causal statistic. All runs use rank-64 LoRA, so the evidence concerns effective-spectrum control under rank-constrained adaptation.

Process reward models (PRMs) are widely used in language-model training with dense, step-level supervision. They assume that PRM scores remain stable proxies for step correctness under label-preserving transformations that alter reasoning structure while preserving final answers. However, this assumption remains insufficiently validated. Such transformations may alter the relationship between PRM scores and correctness, producing different failure modes across models. To address this gap, we introduce EST-PRM, a stress-testing framework for dense process rewards. It evaluates three transformations: (1) step inflation, (2) dependency-aware step reordering, and (3) confidence markers. We further define a vulnerability decomposition that separates reward inflation from loss of correctness sensitivity. We evaluate five PRM-style models on 4,687 reasoning chains from MATH-500, GSM8K, and PRMBench. Results reveal substantial differences in vulnerability patterns across models. Math-Shepherd exhibits the strongest sensitivity to position-based perturbations, with a Pearson correlation drop of 0.139 ± 0.031 and a 29.7 ± 4.2% score inflation rate, while Qwen2.5-Math-PRM is most affected by step inflation, reaching a 47.6 ± 4.3% inflation rate. Confidence-based perturbations further distort reward calibration, revealing inconsistencies in correctness estimation. Finally, we evaluate three mitigation strategies and identify trade-offs between robustness coverage and false-positive rates.

Large language models increasingly require structured inference, from enforcing JSON schema to multilingual parsing, where outputs must satisfy complex constraints. We introduce MetaJuLS, a meta-reinforcement learning approach that learns universal constraint propagation policies applicable across languages and tasks without task-specific retraining. By formulating structured inference as adaptive constraint propagation and training a Graph Attention Network with meta-learning, MetaJuLS achieves 1.5-2.0× speedups over GPU-optimized baselines while maintaining an accuracy within 0.2% of that of state-of-the-art parsers. On Universal Dependencies across 10 languages and LLM-constrained generation (LogicBench, GSM8K-Constrained), MetaJuLS demonstrates rapid cross-domain adaptation: a policy trained on English parsing adapts to new languages and tasks with 5–10 gradient steps (5–15 seconds) rather than requiring hours of task-specific training. Mechanistic analysis reveals that the policy employs human-like parsing strategies (easy-first) and novel, non-intuitive heuristics. By reducing the number of propagation steps in LLM deployments, MetaJuLS contributes to Green AI by directly reducing the inference carbon footprint.

Post-training activation compression is essential for deploying Large Language Models (LLMs) on resource-constrained hardware. However, standard methods like Singular Value Decomposition (SVD) are gradient-blind: they preserve high-variance dimensions regardless of their impact on factual knowledge preservation. We introduce Fisher-Aligned Subspace Compression (FASC), a knowledge-aware compression framework that selects subspaces by directly modeling activation-gradient coupling, minimizing a second-order surrogate of the loss function. FASC leverages the Fisher Information Matrix to identify dimensions critical for factual knowledge, which often reside in low-variance but high-gradient-sensitivity subspaces. We propose the Dependence Violation Score (ρ) as a general-purpose diagnostic metric that quantifies activation-gradient coupling, revealing where factual knowledge is stored within transformer architectures. Extensive experiments on Mistral-7B and Llama-3-8B demonstrate that FASC preserves 6–8% more accuracy on knowledge-intensive benchmarks (MMLU, LAMA) compared to variance-based methods at 50% rank reduction, effectively enabling a 7B model to match the factual recall of a 13B uncompressed model. Our analysis reveals that ρ serves as a fundamental signal of stored knowledge, with high-ρ layers emerging only when models internalize factual associations during training

As the deployment of AI models shifts towards edge devices, developing efficient sequence models has become critical. State-space models (SSMs), particularly Mamba, have emerged as strong rivals to Transformers due to their linear-time complexity and impressive performance across a range of tasks. However, their large parameter counts still hinder their use in resource-constrained environments. To address this, we propose a novel unstructured pruning framework specifically tailored for Mamba, achieving up to 70% parameter reduction with only a 3–9% drop in performance. Unlike pruning techniques designed for Transformers, our approach leverages Mamba’s unique recurrent dynamics by incorporating pruning based on both weight and gradient importance to preserve critical parameters, a gradual pruning schedule to maintain model stability, and a global strategy to optimize parameter allocation across the model. Extensive experiments on the WikiText-103, Long Range Arena, and ETT benchmarks demonstrate significant efficiency gains, including 1.77× faster inference and a 46% reduction in memory usage. Our component analysis confirms Mamba’s robustness to pruning, highlighting the framework’s potential for enabling practical deployment while underscoring the need for careful evaluation to avoid introducing biases in sensitive applications.

Integrating large language models (LLMs) as action proposers in reinforcement learning (RL) significantly boosts performance in text-based environments but incurs prohibitive computational costs. We introduce a cache-efficient framework for Bayesian RL that leverages LLM-derived action suggestions, drastically reducing these costs while maintaining near-optimal performance. Our approach features an adaptive caching mechanism, optimized via meta-learning based on policy performance, to enable efficient inference across text-based games (e.g., TextWorld, ALFWorld) and robotic control tasks (e.g., MuJoCo, MetaWorld). This framework achieves a 3.8×–4.7× reduction in LLM queries and 4.0×–12.0× lower median latencies (85–93ms on consumer hardware), while retaining 96–98% of the uncached policy’s performance. We provide theoretical guarantees on the reliability of cached decisions with Kullback-Leibler (KL) divergence bounds, which are validated empirically by high success rates (90.4–95.6%) in complex text environments. For offline RL, our proposed CQL-Prior variant improves performance by 14–29% and reduces training time by 38–40%. Evaluations across eight diverse tasks demonstrate the framework’s generalizability and practicality for resource-constrained settings, making LLM-guided RL a viable and accessible approach for both text-based and robotic applications.