Medical tasks such as diagnosis and treatment planning require precise and complex reasoning, particularly in life-critical domains. Unlike mathematical reasoning, medical reasoning demands meticulous, verifiable thought processes to ensure reliability and accuracy. However, there is a notable lack of datasets that provide transparent, step-by-step reasoning to validate and enhance the medical reasoning ability of AI models. To bridge this gap, we introduce **MedReason**, a large-scale high-quality medical reasoning dataset designed to enable faithful and explainable medical problem-solving in large language models (LLMs). We utilize a structured medical knowledge graph (KG) to convert clinical QA pairs into logical chains of reasoning, or “thinking paths”, which trace connections from question elements to answers via relevant KG entities. Each path is validated for consistency with clinical logic and evidence-based medicine. Our pipeline generates detailed reasoning for various medical questions from 7 medical datasets, resulting in a dataset of **32,682** question-answer pairs, each with detailed, step-by-step explanations. Experiments demonstrate that fine-tuning with our dataset consistently boosts medical problem-solving capabilities, achieving significant gains of up to **7.7%** for DeepSeek-Ditill-8B. Our top-performing model, MedReason-8B, outperforms the Huatuo-o1-8B, a state-of-the-art medical reasoning model, by up to **4.2%** on the clinical benchmark MedBullets. We also engage medical professionals from diverse specialties to assess our dataset’s quality, ensuring MedReason offers accurate and coherent medical reasoning. Our data, models, and code will be publicly available.

Intelligent workflows for smart manufacturing reliability and maintenance (RxM) are transitioning from static, sequential multi-agent systems toward autonomous, dynamic decision-making. However, a critical gap persists between agent coordination and the holistic co-optimization required across predictive performance, computational resources, and domain-specific business logic. To address this, we propose AgiSphere-CORE, a novel framework centered on a Co-Optimization Core (COC). The COC maintains a unified Collaborative State Vector and solves a constrained optimization problem to maximize the Collaborative Utility J_co, dynamically orchestrating specialized agents for resources, models, and logic. This enables a paradigm shift from sequential to co-optimization-driven autonomy. Extensive evaluation on multiple manufacturing tasks, including classification, regression, and anomaly detection, demonstrates that AgiSphere-CORE consistently outperforms baselines, achieving superior predictive performance (e.g., 97.3% accuracy, 0.902 F1-score), improved resource efficiency (15% faster workflow time), and higher estimated domain cost savings ($148.7K). Ablation studies confirm the critical role of the COC and the adaptive learning mechanism. A detailed case study further illustrates the framework’s transparent, utility-driven trade-offs. We conclude that AgiSphere-CORE provides a robust, holistic foundation for next-generation autonomous RxM systems in smart manufacturing.

Restricting attention to a sliding window makes long documents affordable, but it discards everything beyond the window. This paper asks what can be recovered when the model is already trained, its weights are frozen, and the budget for new parameters is only a few tens of thousands. We propose Multi-scale Spectral Residual Attention (MSRA): a far-field branch grafted onto the last few layers of a frozen model, which condenses out-of-window history through smoothly distance-decayed weights and adds it back to the attention output—roughly 9K–45K trainable parameters, plus minimal 8-to-16-scalar probes for attribution. On PG-19 long-form fiction, the branch closes up to 31.7% of the perplexity gap between local and full attention, with gains stable across two architectures, multiple seeds, and the official test partition. The attribution proves sharper than the headline: 16 learnable output-gain scalars alone close 26.4%, returns accelerate with grafting depth, the carefully designed kernel structure and gating contribute only secondarily—and a budget-matched LoRA closes 87.4%, indicating that the gap is mostly generic capacity rather than a missing far-field pathway. Retrieval experiments complete the map: the correction is a statistical smoother that helps just beyond the window, slightly hurts precise in-window prediction, recovers essentially nothing at genuine long distance, and does not transfer to task metrics—while slowing prefill two-to-three-fold and leaving decoding unaffected. These results bound what a lightweight spectral correction can and cannot recover, forming a reproducible decision reference for local long-context design.

Accuracy remains the dominant criterion for selecting NLP classifiers, yet it fails to characterize the operational reliability required for deployment under class imbalance, distribution shift, and demographic heterogeneity. We introduce TNMM++ (Trustworthy NLP Multi-axis Monitoring, extended), a theoretically grounded reliability audit that evaluates classifiers along seven complementary axes, utility, calibration, robustness, safety, fairness, selective prediction, and uncertainty quantification, and formalises model selection as constrained multi-objective optimization. Beyond metric aggregation, TNMM++ contributes six algorithmic results: a reliability tensor, a Pareto convex-hull theorem, variance-adaptive weighting, a boundary-predictor criterion with exclusion rule for majority-class collapse, a Bayesian risk decomposition, and the TAMS algorithm with optimality bound. Evaluating eight lightweight Transformer encoders (MiniLM, ELECTRA, mBERT, DistilBERT, MobileBERT, ALBERT, RoBERTa, XLM-R) and excluding models that failed to learn the task, we find that properly trained mid-tier models exhibit balanced utility and calibration superior to collapsed high-accuracy counterparts. TrustScore identifies mBERT (0.6685), ELECTRA (0.6630), and DistilBERT (0.6627) as deployment-reliable. Sensitivity analysis under 1,000 Dirichlet weight perturbations confirms ranking stability in 92.4% of configurations, and multi-seed evaluation (5 seeds) demonstrates conclusions are not seed artifacts. Our findings establish that single-metric selection is statistically unreliable and motivate multi-axis reliability auditing as the default protocol for trustworthy NLP deployment.

Fluent LLM explanations may not follow the evidence from a structured system. We present VERITYGATE, a four-gate checker for declared evidence IDs, entities, numbers, and claim types. It checks a fixed schema; it does not verify every fact in the prose. At r=0 and r=1, we test 900 instances per setting (450 grounded–ungrounded pairs) with GPT-4o-mini, Llama-3.3-70B, and Claude Sonnet 4.6. Under this schema-level contract and before repair, 80.3% of mini claims and 47.9% of Sonnet claims fail. These are verifier rejection rates, not prose-hallucination rates. One repair pass raises claim survival from 19.7% to 28.0% for mini and from 52.1% to 54.3% for Sonnet. Verified claims per example change by +0.14 for mini, −0.71 for Llama, and −0.47 for Sonnet, so survival and output volume must be reported together. A second Sonnet pass gives no clear gain. At r=1, Gate 4 covers 97.0%, 98.7%, and 100% of failing claims for mini, Llama, and Sonnet. Small human studies support the rules but show gaps between schema checks and correct prose. A domain-specific GPT-4o judge test shows an order effect, so it is only a usefulness check. We release the code and data.

Retrieval-augmented generation (RAG) grounds large language models (LLMs) in external evidence, yet multi-hop pipelines still suffer from redundant sub-queries, shallow exploration, and premature or delayed stopping. We present EVO-RAG, a phase-aware framework that couples a lightweight two-stage curriculum (Discovery→Refinement) with seven step-level rewards and an in-episode time scheduler. The scheduler decays exploration incentives as evidence accumulates while increasing efficiency and correctness pressure as uncertainty shrinks. Beyond scalar rewards, we train a multi-head preference model and benchmark DPO, PPO, and GRPO under identical rollouts and curricula for a controlled comparison. Evaluated on HotpotQA, 2WikiMultiHopQA, MuSiQue, and Bamboogle with 8B-class backbones, EVO-RAG improves EM/F1 while reducing redundant hops. Ablations show that suppressing query overlap, rewarding controlled backtracking and justified refusal, and time-dynamic weighting are key to the accuracy–efficiency trade-off.

Multi-hop question answering (QA) requires systems to iteratively retrieve evidence and reason across multiple hops. While recent RAG and agentic methods report strong results, the underlying retrieval–reasoning process is often left implicit, making procedural choices hard to compare across model families. This survey takes the execution procedure as the unit of analysis and introduces a four-axis framework covering overall execution plan, index structure, next-step control strategies and triggers, and stop/continue criteria. Using this schema, we map representative multi-hop QA systems and synthesize reported ablations and empirical tendencies on standard benchmarks, including HotpotQA, 2WikiMultiHopQA, and MuSiQue. We highlight recurring trade-offs among effectiveness, efficiency, and evidence faithfulness, and conclude with open challenges for retrieval–reasoning agents, including structure-aware planning, transferable control policies, and robust stopping under distribution shift.

Embodied agents that route every high-level decision through a single language model typically ingest the entire trajectory at every step, a regime in which prompt context grows monotonically and both lost-in-the-middle degradation and trajectory-level hallucination peak. We show that multi-agent decoupling on a shared Voronoi-Frontier blackboard is a structural answer. SkillNav exposes a sparse topology over the partially-built map whose nodes carry visual evidence and two multiplicative gates (mu_safe, mu_mem) written by specialised Safe and Memory agents; a strategic Exploration agent reads only a compact, system-prompt-style summary of the blackboard – whose size is bounded by the topology rather than by trajectory length – and writes back only a phase decision plus an SR/IG weight pair. The three agents never see one another’s prompts; false-positive evidence is suppressed architecturally by mu_mem rather than by asking the language model to revise its own past output. On HM3D val, SkillNav reaches 71.9 SR / 34.0 SPL with only 22.5 LLM calls per episode (~11.9K total tokens), surpassing the strongest published zero-shot baseline (RATE-Nav) by +4.1 SR / +2.7 SPL.

Understanding the geometric structure of pre-trained language model embeddings matters for interpretability and safety. We ask whether sentence-level classification signal lives in the Riemannian geometry of contextual token embeddings, and probe it by extracting per-token pullback metrics from a learned encoder’s analytical Jacobian and aggregating them with the Frechet mean on the symmetric positive definite (SPD) manifold, we call this procedure Riemannian Mean Pooling (RMP). Across three datasets with non-trivial linguistic structure (CoLA, CREAK, RTE), RMP outperforms Euclidean mean pooling, while on FEVER-Symmetric, a benchmark constructed to remove annotation-driven lexical artifacts, the method correctly stays at chance. Ablations show that a randomly initialised encoder combined with Frechet aggregation already beats Euclidean pooling before any training, localising the source of the gain to the geometric aggregation rather than to learned manifold structure. The trained encoder adds a further 1.7 to 2.2 AUC points, consistently across the signal-bearing datasets rather than on any one of them.

Large Language Models (LLMs) have demonstrated remarkable fluency and versatility across a wide range of NLP tasks, yet they remain prone to factual inaccuracies and hallucinations. This limitation poses significant risks in high-stakes domains such as healthcare, law, and scientific communication, where trust and verifiability are paramount. In this paper, we introduce DAVinCI - a Dual Attribution and Verification framework designed to enhance the factual reliability and interpretability of LLM outputs. DAVinCI operates in two stages: (i) it attributes generated claims to internal model components and external sources; (ii) it verifies each claim using entailment-based reasoning and confidence calibration. We evaluate DAVinCI across multiple datasets, including FEVER and CLIMATE-FEVER, and compare its performance against standard verification-only baselines. Our results show that DAVinCI significantly improves classification accuracy, attribution precision, recall, and F1-score by 5-20%. Through an extensive ablation study, we isolate the contributions of evidence span selection, recalibration thresholds, and retrieval quality. We also release a modular DAVinCI implementation that can be integrated into existing LLM pipelines. By bridging attribution and verification, DAVinCI offers a scalable path to auditable, trustworthy AI systems. This work contributes to the growing effort to make LLMs not only powerful but also accountable. Our code is available at https://github.com/vr25/davinci.

Language models (LMs) are increasingly used to generate decisions and explanations for high-stakes financial tasks, but these explanations can contain factual inconsistencies with the underlying claim or invalid causal reasoning. We propose a two-step auditing framework that decomposes each explanation into reasoning points and verifies them using natural language inference (NLI): (i) claim ⇒ point for factuality verification and (ii) point ⇒ conclusion for causality verification. To reduce reliance on expensive expert annotations, we study effective fine-tuning strategies based on semi-supervised learning (SSL) for NLI verifiers, leveraging abundant unlabeled explanations via pseudo-labeling and FreeMatch with self-adaptive confidence thresholds. Experiments across small language models and both encoder-only and decoder-only verifiers under varying labeling budgets show that SSL consistently outperforms the pre-trained baseline and supervised fine-tuning. FreeMatch is generally more robust than simple pseudo-labeling, suggesting better utilization of unlabeled data for scaling error detection of LMs’ generations.

Open-weight language models remain modifiable after release, so release-time refusal alone cannot establish durable safety. We introduce removal-cost grounding as a budget-conditioned audit of whether safety can be weakened while preserving utility, coherent generation, and benign helpfulness. We formalize clean removal as a constrained feasibility problem and distinguish observed counterexamples from inconclusive attack searches. Mycelium v0 provides a deliberately limited case study: four model families exhibit benchmark degradation under an archived removal configuration, while targeted Mistral controls separate destructive ablation from wrapper bypass. A provenance audit excludes three model–task comparisons with mismatched evaluation limits or conflicting snapshots. Across the retained comparisons, mean removal-associated score changes range from -0.382 to -0.185. In a separate 12-prompt generation panel, the strict attack-success proxy increases from 1/12 with Mycelium enabled to 7/12 under bypass, with wide uncertainty and no matched native safety baseline. Adaptive traces further show why early-stopped checkpoints must be distinguished from training endpoints. These observations do not establish tamper-resistant safety. They motivate an auditable reporting framework in which the cheapest admissible route to useful unsafe behavior, rather than the costliest destructive ablation, determines the strength of a safety claim.

Retrieval-Augmented Generation (RAG) improves factual grounding, but it also lengthens prompts and raises prefill cost. Prefix caching in serving engines such as vLLM reduces this cost only when requests share the same token prefix. In grounded generation, however, adjacent queries may retrieve overlapping evidence in different orders, so set overlap does not become reusable prefix overlap. We present CacheWeaver, a lightweight prompt-layer method for cache-aware evidence ordering. The method keeps a prefix tree over recently served evidence sequences and uses a greedy walk to place the most reusable prefix first, while leaving the serving engine and retrieved evidence set unchanged. Across three vLLM configurations the method lowers median time-to-first-token (TTFT) by about 20 to 33% relative to retrieval-order prefix caching, without hurting answer quality in our QA tests. The greedy policy reaches 97.5% of the median TTFT gain from oracle ordering, which indicates that most reusable prefix locality can be recovered by a simple scheduling layer between retrieval and inference.

We provide a confound-controlled methodology and a released "wrong-question" diagnostic for internals-based, reference-free hallucination detection, which treats context-faithfulness (is an answer entailed by the context?) as a linear residual-stream direction. In the standard extractive setup, faithfulness is almost perfectly correlated with lexical overlap (r=0.99), so an apparent "grounding probe" may be a string-matching detector. Our matched Support×Overlap construction partially decouples the two (halving the confound to r=0.50), and the diagnostic fools an off-the-shelf NLI model. Across four instruction-tuned 7–9B families: (i) within the overlap-controlled stratum a support contrast is linearly decodable, but a no-context ablation shows it reflects question–answer compatibility rather than context use, and is inseparable from factoid-correctness by construction; (ii) it aligns across families up to a linear map, yet answer-length, overlap, and factuality directions align equally well, a generic matched-stimulus property, not grounding-specific; (iii) on the three families we could extract, the synthetic direction does not transfer to real hallucinations, scoring below NLI, confidence, and lexical overlap on RAGTruth, where only an in-domain probe trained on human labels recovers a usable signal.

Prompt injection defenses evaluated on synthetic benchmarks do not generalize to real enterprise documents, which are longer, denser, and interleave legitimate authority language with factual content. We demonstrate this gap with a benchmark of 122 tasks across five professional domains (financial, legal, medical, scientific, DevOps) built on real retrieved documents—actual SEC filings, Federal Register rules, PubMed abstracts, arXiv papers, and GitHub postmortems—paired with LLM-generated tasks and camouflaged payloads that were not human-validated. Paraphrasing, the strongest defense on synthetic benchmarks, shows no statistically significant attack success rate reduction on real documents (p=0.500) while degrading utility from 91.8% to 82.8%. We introduce PARSE (Provenance-Aware Retrieval Sanitization), a domain-aware, fact-preserving sanitization pipeline that classifies each sentence by injection likelihood, extracts structured facts before rewriting, and verifies fact preservation via a consistency-checking loop. A directiveness gate routes 59% of real enterprise documents to a lightweight path, concentrating computational cost on high-risk documents. PARSE achieves 15.6% attack success rate—a 39% reduction versus the 25.4% baseline—at 86.9% utility, the largest reduction of any condition evaluated (h=−0.245), nominally significant (p=0.014) but not surviving correction for multiple comparisons. Practitioners should evaluate defenses on domain-matched real documents, not synthetic proxies.

Weight quantization is the default route to running language models on consumer hardware, yet its effect on contextual faithfulness—a model’s willingness to ground answers in provided context rather than memorized priors—remains untested: faithfulness benchmarks use full-precision models, and quantization studies measure accuracy and perplexity, not grounding. We audit two instruction-tuned 3B models (Qwen2.5-3B, Llama-3.2-3B) across seven GGUF precisions from 16-bit to 2-bit, under aligned, counterfactual, and unanswerable contexts, using 300 SQuAD-derived items and three prompt templates per condition (900 trials per condition per precision). With an item-level clustered bootstrap we find faithfulness unchanged from 16 down to 5 bits in both models, while below that small but statistically detectable declines appear whose onset is model-dependent (4-bit for Llama, 3-bit for Qwen). Critically, production of the original, context-contradicting answer does not rise at any precision, so the intuitive "quantization erodes grounding" mechanism is not supported: lost faithfulness moves to off-target responses instead. Qwen additionally collapses into incoherent output at 2-bit while Llama stays fluent, making extreme compression a per-model question. Code and harness: https://github.com/movvaneeraj/quantfaith.

Grounding faithfulness is measured on two axes: whether the answer follows the retrieved context, and whether each claim is supported by the source it cites. Both can be satisfied in full by an answer that picks one side of a genuinely contested question, cites a source that holds that side, and drops the credible sources that disagree. We name this failure disagreement collapse, or the citation-faithfulness illusion, and treat it as a construct-validity hazard: a high citation score does not license the inference that the answer reflects the sources. Conflict-aware retrieval already prescribes reflecting credible debate; our contribution is to make that prescription measurable. We define disagreement recall and disagreement-attribution precision, both decoupled from answer correctness and conditioned on source credibility so that suppressing a debunked claim is not penalized; a five-component rubric with verbatim anchors; an annotation protocol for the credibility filter and for position identity; a matched-pair diagnostic that holds retrieval and citation availability fixed; and a pre-registered two-judge pilot with human adjudication. No empirical scores are reported; the contribution is the instrument and its frozen evaluation design.

Retrieval-augmented generation treats citation as a default proxy for reliability, yet a cited passage can support a claim and still be obsolete, superseded, contradicted, or invalid at the user’s target time. We name this failure a citation-supported lifecycle error and define lifecycle faithfulness: a claim-level criterion that requires semantic support, target-time validity, non-supersession, and provenance. We release Vera-Life-Eval, an 8,500-question benchmark with 41,203 versioned evidence pieces across six lifecycle settings (current, past, stale-only, update, conflict, and multi-hop), and Vera-Life, a low-training modular controller that builds an evidence-state graph, selects claim-evidence-time contracts, and chooses among answer, warn_stale, explain_conflict, retrieve_more, and abstain, emitting a claim-level lifecycle ledger. On a shared Qwen3-8B generator, Vera-Life raises LifeEM from .374 (best prompt-only baseline) to .623, and remains effective under timestamp-only (.584) and text-only (.504) regimes. Same-action controls, metadata isolation, and source-disjoint transfer across product, policy, scientific, and news evidence show that the gains are not an artefact of output format or Wikipedia revision cues. The next mission for RAG is not merely to cite evidence, but to know whether that evidence is still alive.

Practice often treats automatic metrics for attribution in LLM retrieval-augmented generation as interchangeable. We audit eight automatic scorers — lexical, embedding, and BERTScore baselines alongside entailment/grounding-trained models (clean and FEVER NLI, the checker MiniCheck) — across three evaluation constructs (provenance/topicality, generated-answer attribution, and fact-check entailment), asking whether any scorer transfers: stays within the 95% confidence interval of the best audited scorer on every dataset of a multi-dataset construct. In the construct with the most multi-dataset human-labeled coverage — generated-answer attribution (AttributionBench’s four source datasets, n = 1,610, with independent HAGRID, n = 2,150) — none of the audited automatic scorers does: the per-dataset metric rankings invert (Kendall τ = −0.64, p = 0.031 on AttributedQA vs. LFQA), and an off-the-shelf NLI scorer that is best on short-claim AttributedQA (AUROC 0.90) collapses to AUROC 0.53 (chance) on long-form LFQA, where BERTScore wins (0.91); the reversal persists under the tested truncation settings. This instability has a concrete decision cost: a naive "best-on-average" rule for choosing an evaluator fails leave-one-dataset-out (mean held-out regret 0.172 AUROC, worse than fixing one scorer), so metric choice should be validated on the target dataset rather than assumed from performance elsewhere. A prompt-based LLM judge avoids the chance-level collapses the automatic scorers suffer (no LFQA collapse) but is not uniformly best, ~100× costlier, and non-deterministic — relocating, not removing, the validation burden.

Grounded language-model systems are often evaluated by final answer accuracy, yet a correct answer can be unsupported, drawn from the wrong source, or produced when evidence is insufficient or contradictory. We introduce EviScope, a paired counterfactual benchmark that holds the question fixed while adding, removing, distracting, or contradicting its evidence. EviScope-v1.1 contains 40 four-condition quartets with repaired counterfactual claims and span-level support labels for automatic evaluation. Across 960 gold-blind generations from Qwen2.5-7B, Llama 3.1 8B, and Gemini 3.5 Flash, paired metrics expose model-dependent grounding behavior that answer accuracy hides. On two local open models, an explicit evidence-action gate underperforms vanilla RAG on QCS: 0.15 vs. 0.50 for Qwen and 0.10 vs. 0.375 for Llama. Gemini reaches 0.944 joint success under both prompts, yet still answers 5% of conflict cases after contradiction insertion. EviScope therefore distinguishes unsupported answering, conflict blindness, and wrong non-answer actions rather than scoring answers alone.

Retrieval-Augmented Generation (RAG) is usually evaluated by whether the final answer is correct. Under knowledge conflict, this hides a key question: did the model follow retrieved evidence, rely on its parametric prior, or produce a post-hoc rationale? We study this as context compliance, the regime in which retrieved context controls the answer even when it conflicts with the model’s prior knowledge. We introduce Context-Driven Decomposition (CDD), an inference-time diagnostic intervention that elicits contextual and prior answers, isolates the conflicting premise, and records a resolution trace that can be perturbed. Across Epi-Scale stress tests, TruthfulQA misconception injection, and cross-model reruns, CDD makes three behaviors visible. First, misleading retrieval can severely degrade accuracy: under worst-case injected context (a TruthfulQA misconception-injection probe), Standard RAG reaches only 15.0%. Second, better answers need not share the same mechanism: CDD improves adversarial accuracy on Gemini-2.5-Flash and shows directional gains across Claude variants, yet trace-perturbation sensitivity is high only on Gemini. Third, explicit decomposition improves controlled-conflict robustness over a conflict-aware instruction baseline on localized factual conflicts, with the clearest margins on Entity Swap (88.0% vs 79.3%) and Logical Contradiction (83.2% vs 75.4%). We frame RAG conflict handling as an observability problem.

In medical LLM adaptation, collecting large-scale real clinical notes is constrained by patient privacy and annotation cost, so training pipelines often rely on notes synthesized by commercial LLMs. Those synthetic notes are not error-free. We observe that after fine-tuning on such notes, a task judge rates the resulting model’s outputs as more complete without a consistent Accuracy gain, while a hallucination judge flags more statements that the source does not support. We therefore report Accuracy and Completeness separately from hallucination counts, split the latter into Content and Format-Meta hallucinations, and report both as rates normalized by the number of fields each output format requires, so that a gain in one cannot hide a loss in the other. We then introduce Self-Detect–Refine, which applies an external LLM to detect and revise hallucinations in the training labels themselves rather than in outputs at inference time. We evaluate it across three base models, two refiners, and four refinement depths, fine-tuning a fresh copy of each base model on each label snapshot. GPT-5.2-refined labels lower Content hallucinations in every configuration, both as a share of the required fields and after length normalization. The two stages load on different components: the Completeness gain belongs to raw-label fine-tuning, whereas refinement raises Accuracy without a consistent loss in Completeness. Format-Meta changes are mixed, a lower-cost refiner is less consistent, and every judgment here comes from LLM judges on a single synthetic benchmark without clinician validation.

Graph-grounded language models are usually evaluated by answer accuracy, yet a serialized prompt is a string representation of a graph rather than the graph itself. We argue that faithful graph-grounded reasoning requires the composed predictor F𝜃 ∘ z to be well-defined on graph-query equivalence classes: structurally equivalent graph-query pairs should induce the same prediction regardless of textual realization. We formalize this as a structural invariance property, decompose unfaithfulness into syntactic instability, semantic interference, and distractor susceptibility, and introduce corresponding diagnostics. On controlled graph reasoning tasks, DeepSeek-V4-Flash achieves 98.9% accuracy and near-perfect syntactic stability (FS = .975), yet its isomorphism consistency drops from .969 to .701 under semantic-name relabeling (∆_sem = .268); only 72.9% of originally correct answers survive this transformation. Across four model families, we observe targeted semantic interference in high-accuracy models, broad unfaithfulness in weaker models, and near-perfect invariance as a capability ceiling. A graph-grounded LLM can be accurate without being structurally faithful: its predictions may fail to remain invariant under graph-equivalent textual relabelings.

Multi-turn scheduling assistants must preserve constraint edits and distinguish solver evidence from language-model interpretations. We present CoSched-Cert, a dialogue protocol combining a typed scheduling language, solver-distinguishing clarification, cost-based conflict localization, verified repair synthesis, and lexical certificate auditing. Its guarantees are conditional on the encoded problem and supplied candidate interpretations; lexical support does not establish semantic entailment. SchedNL-Bench-Cert contains 1,000 synthetic Chinese/English utterances and 350 sessions. Gold-AST replay measures internal consistency against labels generated by the same engine, while author-written protocol simulations illustrate selected design choices. Across three live-parser seeds on 52 sessions, session training and registered sibling augmentation raise conditional micro target-constraint agreement from 18.0% to 83.7%. The final configuration has 100% coarse status agreement but only 75.0% conflict-member agreement, and includes a rule informed by a test-session failure. We therefore treat these results as descriptive, not as an unbiased generalization estimate. We define a stricter joint-turn criterion and report the bounds supported by the available aggregates. The contribution is a protocol for exposing inspectable solver evidence and the remaining gap between encoded feasibility and user-intent correctness.

Multi-hop retrieval-augmented generation systems often produce answers with incomplete or noisy citation sets. When the answer text is fixed, citation repair may seem to require only adding retrieved evidence. We show that this view is incomplete: in multi-hop settings, expansion that recovers missing support can also introduce plausible distractors into the evidence bundle. We study fixed-answer citation-set repair as a diagnostic protocol for reporting policies as operating points between support-ID coverage and distractor control. Across a calibrated 5,000-example HotpotQA diagnostic and Qwen2.5-generated outputs on HotpotQA, MuSiQue, and 2Wiki, cross-encoder repair yields cleaner support-ID operating points than BM25 expansion on HotpotQA and 2Wiki, while MuSiQue shifts the policy into a higher-distractor-cost regime under harder compositional transfer. We do not claim universal transfer of absolute performance from HotpotQA alone; the cross-dataset pattern is instead consistent with mechanism-level transfer of the support-distractor operating-point geometry. A single-author audit of 450 unique generated-output cases plus 60 duplicate checks reinforces this caution: support-ID coverage and claim-level citation support can diverge. We therefore recommend an operating-point scorecard for citation repair and separate support-ID diagnostics from stronger claim-level faithfulness claims.

Customer-service QA in an AI contact center (AICC) runs under deployment constraints that benchmark QA misses: tight voice-hotline latency and a high cost for unsupported or wrong automatic answers. We deploy a system that answers only from a closed set of verified QA units: it returns a retrieved unit verbatim, or routes to clarify/abstain/handoff.The index is enriched offline by staged linguistic seeding (SLS): a human authors a per-unit world-grounded slot recipe, gpt-4.1-mini renders it into variants, and a light human gate filters them, one methodology reused across both domains, so inference stays a single retrieval pass with no query-time generation.On held-out query variants (in-distribution recipe reformulations) from two industrial domains, SLS is the dominant retrieval lever, lifting hybrid R@1 to 0.881/0.930 (+0.27/+0.34); the gain holds across all five retrievers we test. The win is the design, not the volume: at the same gpt-4.1-mini, SLS beats a budget-matched doc2query by +0.20/+0.32, and a cross-provenance control provides additional evidence of transfer across generated-query distributions; the embedding choice is secondary. Verified-unit answering also removes free-form generation’s unsupported-content surface (7–13% vs. ≈0%). We report this as an honest application study, including negative results.

Structured pruning has gained attention as a promising solution for the cost-effective deployment of large language, vision-language, and domain-specific foundation models. However, most prior work has focused on general-purpose language models and often lacks a systematic comparison of pruning strategies across model scales, architectures, and real-world multimodal applications. In this paper, we conduct a comprehensive evaluation of structured pruning methods, focusing on depth pruning, width pruning, and post-pruning recovery with parameter-efficient fine-tuning. We evaluate multiple open-source models, including Qwen3-0.6B, Qwen3-4B-Instruct, Qwen3-8B, LLaVA-OneVision-0.5B, and Phi-4-Mini-Instruct, across intrinsic metrics and downstream benchmarks such as BBH, MATH Hard, MuSR, and MMLU-Pro. Our findings reveal that (1) depth pruning generally provides a more stable trade-off between efficiency and performance than width pruning; (2) width pruning can reduce memory usage, but often gives limited throughput gains and less predictable accuracy retention; (3) reasoning-intensive benchmarks, particularly mathematical reasoning tasks, are more sensitive to structured pruning than some other evaluation tasks; (4) LoRA-based post-pruning adaptation can substantially recover performance after moderate pruning; and (5) pruning behaviors observed in general-purpose models transfer to Earth observation foundation models, as demonstrated on TerraMind, where depth pruning reduces latency and GPU memory usage while maintaining competitive EuroSAT and Bigearthnet task accuracy.

Retrieval-augmented language models can fail to respect negative constraints when the retriever supplies evidence about concepts the user explicitly excluded. Beyond explicit negation, queries may ask for answers that include one concept while excluding another, or for entities that belong to a category but differ from a closely related instance. Because the excluded concept still appears in the query text, dense retrievers may assign high similarity to documents about that concept even when the user asks to avoid it. We introduce E-SENS, a training-free reranking method for negation-sensitive retrieval. E-SENS extracts a compact trap query for the excluded side and subtracts trap-query similarity from the original-query retrieval score. On ExcluIR, E-SENS shows a clear recall-violation trade-off across four embedding models and reduces trap retrieval at recall-preserving settings.

Hallucination detection has become a pressing requirement for trustworthy AI deployment at scale. The most accurate detection methods depend on GPU-intensive inference, proprietary API calls, or white-box access to the generating model. This puts them out of reach for resource-constrained researchers and practitioners. In this paper, we explore a practical alternative: how well can hallucination detection perform using only lightweight, CPU-feasible methods built on publicly available models? We systematically benchmark four such detectors, ROUGE-L, semantic similarity, BERTScore, and a Natural Language Inference (NLI) detector based on a FEVER-trained DeBERTa model, together with a score-level ensemble of similarity and NLI. We evaluate them across all three tasks of the HaluEval benchmark: question answering (QA), dialogue, and summarisation. We calibrate each method on a held-out validation split and evaluate it on 2,000 test instances per task, and we report bootstrap confidence intervals for the main results. We find that the similarity-NLI ensemble is the most consistent method, but that absolute performance is highly task-dependent. The ensemble ranks best on QA (F1 = 0.792, AUC-ROC = 0.873) and on dialogue (F1 = 0.694, AUC-ROC = 0.749), where NLI is the strongest standalone method; on summarisation every method performs near chance (AUC-ROC between 0.469 and 0.574). We then ask whether that failure is intrinsic to lightweight detection or an artifact of our single-pass design, and find it is largely the latter. Raising the premise budget from 800 to 1600 characters lifts summarisation AUC-ROC from 0.567 to 0.629, and replacing single-pass scoring with sentence-level chunk aggregation reaches 0.683, still on CPU and with the same model, though at roughly twenty times the NLI inference. Summarisation remains by far the hardest task, but our results do not support treating lightweight detection as intrinsically unsuited to it. All experiments run on a standard laptop CPU using public models.

Memory is a critical component of modern conversational systems. Prior research on memory mechanisms in RAG-based conversational systems has focused on how memory is stored and retrieved. However, less is known about how memories with different functional roles influence response quality, shape agents’ responses across conversational contexts, and lead to distinct response behaviors. Existing evaluations are also largely reference-based, often missing response nuances that address users’ preferences differently. In this work, we study the impact of different memory types on agent responses. We present a fine-grained taxonomy of conversational memory, classify retrieved memories by functional role, and design a user-centric evaluation framework that simulates user perspectives. Through comparative experiments on long-term datasets and frontier LLMs, our analysis reveals differentiated effects of memory: e.g., clarifying memory is associated with improved factual accuracy and constraint awareness, making responses more correct and personalized, while irrelevant memory reduces topic relevance and degrades constraint awareness. These findings show that, despite the power of frontier LLMs, different memory types can be leveraged to produce responses better aligned with a user’s stated constraints, and motivate further research in this direction.

We diagnose relational grounding failures in zero-shot English-to-Chinese dialogue summarization with small language models (SLMs). We develop and apply a Dialogue-Semantic Grounding Error Taxonomy to evaluate three open-weight SLMs alongside a supervised mBART baseline. The taxonomy captures errors in entities, events, speaker roles, circumstances, modality, discourse outcomes, omissions, and language or format issues. Our analysis shows that SLM hallucinations often involve relation-sensitive distortions of dialogue-relevant content: models may preserve source-relevant participants, events, or circumstances while altering who did what, under what conditions, or with what outcome. These findings position the taxonomy as a diagnostic evaluation framework for dialogue summarization, revealing relation-sensitive faithfulness failures that reference-based metrics may obscure.

Self-distillation for grounded question answering often filters tutor trajectories with agreement or confidence gates. For such gates to support faithful grounding, their signals should depend on the supplied document rather than on answers recoverable from parametric memory. We propose a four-mode randomization-control diagnostic that applies the same frozen gate to original, randomized, irrelevant, and empty documents for the same question. On HotpotQA and MuSiQue, using Qwen2.5-7B-Instruct with eight tutor traces, the joint gate changes sharply with document condition: kept fraction falls from 0.380 to 0.020 and from 0.330 to 0.015, respectively. Randomized and irrelevant documents lie between the endpoints, while a document-blind random gate stays flat. The protocol audits document sensitivity without additional student training; faithful evidence use and downstream gains require separate validation.

Retrieval-Augmented Generation (RAG) pipelines often pass a fixed number of retrieved documents to the generator, even though evidence needs vary by query. We study context selection as a post-retrieval control problem: given the same candidate list, how much evidence should enter the prompt, and in what form? We present Safe Adaptive Context, a lightweight controller that starts with limited evidence and expands when the generated answer appears weak, and TACER, a task-aware extension that routes between compact evidence and safer broader context using retrieval-side and evidence-coverage signals. Across five datasets, two generators, and stronger retriever settings, the controllers substantially reduce provider-reported token use while preserving most Fixed Top-10 answer quality. In a five-dataset, 500-query external-baseline stress test, Safe Adaptive Context retains 99.8% of Fixed Top-10 F1 with 55.0% token reduction, while TACER retains 99.4% with 60.4% token reduction. The contribution is not universal quality improvement, but practical quality-preserving efficiency through task-aware context control.

Agents grounded in heterogeneous sources often encounter conflicting evidence. We study an Audited Reasoning Pipeline (ARP): a forensic analyzer reads the raw sources and emits a sanitized diagnostic, while a clean-room synthesis stage receives the question and diagnostic but not the raw context. On 100 corrupted ConflictQA instances per model, ARP raises exact match from 15 to 30 for GPT-OSS-120B and from 19 to 29 for MiniMax-M2.5.CAT-Source changes from 40 to 49 and from 74 to 56, respectively; their paired model-by-condition interaction is 27 points (95% CI [13,41], p < .001), although two model/configuration pairs do not identify capability as its cause. The three diagnostic metrics are emitted before the clean-room boundary, and all items are corrupted, so conflict detection has no false-positive evaluation. Token-F1 also falls on multi-answer questions. Because the clean four-call control that gives Stage 4 both raw context and the diagnostic was not run, the air- gap’s causal effect remains unidentified.

A growing line of work tests whether large language model (LLM) explanations are faithful by intervening on the explanation (removing, perturbing, or swapping it) and checking whether the model’s final answer changes – an Answer Change Rate (ACR), read as causal evidence. We argue this inference requires a control existing work has not consistently reported: whether the answer changes comparably under an unrelated disturbance. We run the causal intervention paradigm on 12 models (6 matched base/instruction-tuned pairs, 8B–70B parameters) on GSM8K and MMLU-STEM, adding a content-free filler control matched for length, a re-injection baseline measuring re-prompting and decoding instability, and paired significance testing across conditions. McNemar’s test does not detect a significantly larger answer-change rate for removal than for the control in 14 of 24 model–task cells (58%) – an absence of detected difference, not equivalence – the re-prompting/decoding floor alone exceeds 80% answer-change in one model, and a commonly assumed instruction-tuning effect does not replicate across families. We do not conclude that LLM explanations are faithful or unfaithful; we conclude that ACR frequently fails a basic specificity check and cannot alone support causal-faithfulness claims. We release the full pipeline and propose a minimal reporting standard for future studies.

Post-training has become the primary mechanism for grounding large language models (LLMs) to application-specific domains, enabling them to acquire new knowledge, behaviors, and policies beyond pre-training. Supervised fine-tuning (SFT) and preference optimization are the dominant paradigms for this adaptation, but are typically applied sequentially, requiring careful tuning of training schedules, regularization, and other hyperparameters.

We propose Gradient-Aligned Iterative Fine-Tuning (GAIF), an adaptive post-training framework that combines three complementary components: a unified SFT+DPO objective optimized simultaneously; a prospective adaptive weighting mechanism that selects task weights based on next-step gradient geometry; and an iterative outer loop that regenerates preference pairs from the current model’s failures, keeping the supervision signal synchronized with the model’s evolving capabilities while adapting training toward arbitrary evaluation metrics, including non-differentiable ones.

We evaluate GAIF on a production-scale generative movie search assistant, where a 7B LLM is adapted to the entertainment domain while remaining faithful to a fixed streaming catalog. The 7B model trained with GAIF outperforms standard SFT, sequential SFT+DPO, and a ~90B reference model in both ranking quality (NDCG) and output validity. Experiments on GSM8K mathematical reasoning further show the algorithm’s effectiveness extends beyond grounded retrieval.

Compressing long multi-turn conversations requires deciding not only how to compress context, but which signal should rank memory candidates under a fixed budget. We study a sparse keep/compress/drop (KCD) codec and compare geometric, semantic, and hybrid scoring signals across conversational memory regimes. On a stress-test benchmark designed to expose support-turn failures, a geometry-scored codec substantially outperforms a semantic signal swap inside the same codec. On MSC (1,000 conversations), we show that geometry-based compression methods substantially outperform LongLLMLingua on geometric fidelity (ΔL2 ≈ +130) while LongLLMLingua modestly improves answer NLL—a geometry-behavior divergence confirming that behavioral metrics alone are insufficient to evaluate memory compression. Among KCD signal variants on MSC (1,000 conversations, Qwen2.5-1.5B), query-conditioned geometry achieves the best combination of geometric fidelity and behavioral quality simultaneously: at budget 0.50 it reduces logit L2 by 109.4 units versus semantic KCD (p<0.001, conversation-level) while also achieving best answer NLL across all budgets. On LongMemEval-S (100 full-length conversations), all KCD variants significantly improve behavioral quality over uniform compression. A Llama-3.2-3B cross-model check shows that the geometry signal advantage is model-family-dependent: semantic compression dominates geometry compression on behavior across all budgets for Llama-3B (p≤0.01, conversation-level). The paper’s central claim is that geometry-behavior divergence is real and consequential, and that query-conditioned geometry is the signal that best resolves it on Qwen2.5 models.

Retrieval-augmented generation (RAG) systems can precompute and store key-value (KV) caches of retrieved documents to avoid re-encoding the same context for every query. Quantizing these offline caches reduces storage cost, but existing work largely evaluates compression through accuracy alone and does not ask whether the generated answers remain faithful to the retrieved evidence. This distinction is important because a model can preserve the correct answer string while weakening its grounding in the provided context. We audit Qwen2.5-7B-Instruct under BF16, INT8, and INT4 offline KV-cache settings on RGB and HotpotQA, measuring both answer accuracy and faithfulness using a hallucination detector, NLI entailment, and an LLM judge. INT8 is nearly lossless across accuracy and faithfulness metrics. In contrast, INT4 reduces accuracy and, more critically, degrades faithfulness even on examples where the answer remains factually correct: over 90% of faithfulness flips are negative. The degradation increases under noisier retrieval and larger retrieval depth. These results show that compressed RAG systems cannot be validated by EM or F1 alone; faithfulness must be audited before low-bit offline KV caches are deployed.

We introduce a novel approach for calibrating uncertainty quantification (UQ) tailored for visual language models (VLMs). Existing state-of-the-art UQ methods rely on consistency among multiple responses, but often report higher confidence when the model is consistently incorrect, leading to poorly calibrated confidence. To address this, we leverage cross-modal consistency by grounding back the textual responses to visual inputs. The confidence from the grounding model (GM) is used to calibrate the overall confidence. Given that using a GM adds its own uncertainty, we apply temperature scaling to calibrate the GM’s confidence in the accuracy of the generated responses. We comprehensively evaluate the proposed approach across diverse range of settings, including datasets with varying visuo-linguistic complexity, Q&A and caption generation tasks, general visual and specialized medical contexts, VLMs differing in scale, openness, and different categories of GMs. Experimental results show that our approach consistently improves confidence calibration, with expected calibration error (ECE) improvements ranging from 32.5% to 96.7% over both self and non-self consistency-based baselines. Code is available here https://github.com/trilokpadhi/Multimodal-Uncertainty-Quantification.

We introduce CrossCTI, a multilingual benchmark for evaluating retrieval-augmented generation (RAG) grounding faithfulness in open-source large language models applied to cyber threat intelligence (CTI) analysis. Existing CTI and forensic NLP benchmarks are predominantly English-centric, leaving a critical gap in understanding how grounding faithfulness degrades under cross-lingual distribution shift, a realistic condition in multinational security operations centers. CrossCTI translates 50 English CTI reports into Arabic, Chinese, Swahili, and Russian using NLLB-200 with entity-preserving masking, achieving 100% IOC preservation across languages. We construct 26 forensic question-answer pairs and evaluate each against all five language corpora and three open-source models (Qwen2.5-7B, Llama-3.1-8B, and Mistral-7B-v0.3), producing 390 evaluation instances, scored with token-level F1, character-level F1, and entailment-based attribution. Using a repeated-measures mixed-effects analysis that accounts for the fact that each of the 26 items is evaluated under all 15 model×language conditions, we find that cross-lingual degradation is severe, but uneven across metrics: language is the dominant, robust predictor of lexical (token-level) grounding quality (p<0.001), with no detectable effect of model identity on this metric. In contrast, model identity is a significant predictor of both character-level overlap and entailment-based attribution (p<0.01), including a specific interaction in one model (Mistral) that underperforms specifically in Swahili. Token-level F1 drops from 0.44 (English) to 0.03 (Arabic); gold-chunk retrieval recall drops from 0.962 (English) to 0.115 (Arabic); and the instance-level correlation between retrieval success and token-level grounding is r=0.60. Performance also improves from the top-k=1 to the top-k=3 retrieved passages before plateauing, with no consistent gains from the additional retrieved content. We further show that the entailment-based attribution metric is significantly elevated specifically for the model used to generate the benchmark’s own QA pairs, a synthetic-annotator confound that we quantify directly via within-item paired tests rather than only disclosing qualitatively. CrossCTI provides a reproducible framework for multilingual CTI grounding evaluation and a case study in how the apparent dominant factor, language or model, depends on which grounding metric is used.

We present RuleChef, a framework that uses large language models (LLMs) to generate executable rules for NLP tasks such as text classification, Named Entity Recognition (NER), or relation extraction. Rules are generated based on a task description and a set of labeled examples, then they are iteratively improved based both on additional examples and on human feedback over existing rules. RuleChef can also be used to bootstrap rules using the observed input-output pairs from any existing model for a given task. LLMs are used only at learning time, synthesizing rules and iteratively patching them based on failures measured on a held-out split. The result of this process is a fast, deterministic, and inspectable rule system. Preliminary evaluation is performed on both classification and NER tasks. We release RuleChef as open-source software under an Apache 2.0 license.

KV-cache compression reduces LLM inference memory by evicting context tokens, but when the evicted tokens contain answer-bearing evidence, the model may hallucinate instead of recognizing that the compressed context is insufficient. We address this failure from a behavioral perspective: to our knowledge, this is the first work to formulate compression-aware abstention as a learning problem, in which a model learns to answer when supporting evidence survives compression and abstain when it does not. We construct supervision from compressor survival masks and tight answer-bearing spans, labeling examples as Confident when evidence survives and Abstain when it is removed. A 10.1M-parameter LoRA adapter trained on ∼2.6K MuSiQue 2-hop QA examples reduces base-model hallucinations by 97% under prompt-style truncation while preserving correct answering on evidence-retaining examples. Unlike prompt-only abstention baselines, which over-abstain on many answerable high-retention examples, the trained adapter learns a conditional policy. We also evaluate the method under actual compressed-cache decoding, where multi-compressor training yields a 6–22× relative lift over the unaided base on evidence-retaining examples. Controlled-deletion experiments show that the learned behavior is driven by evidence content rather than input length alone.

LLM-as-a-judge evaluation is widely used for retrieval-augmented generation (RAG), but reusing a model as generator and judge complicates identifying self-leniency. We introduce Eval-Pair Matrix, a controlled meta-evaluation protocol for source-grounded RAG. Starting from GaRAGe, we induce one hidden answer-causal contradiction per record, generate answers from perturbed passages with GPT, Grok, and Gemini, and use the same models as blind judges against the original passages. The experiment contains 300 core records, 897 labeled outputs, and 2,683 verdicts in a crossed 3×3 matrix; the primary analysis uses 275 validated records. We estimate same-model effects by pairing judges on the exact same candidate answer. Diagonal and off-diagonal F1 are similar, and aggregate paired recall is near zero for the three tested endpoints (−0.5 pp; 95% cluster-bootstrap CI [−2.7,+1.7]). Matching judges flag answers that avoided the induced claim less often (−4.3 pp). A targeted human evaluation classifies reviewed apparent false positives as alternate source-error detections, adoption-label mistakes, or unclear cases; none were adjudicated as genuine false alarms. We recommend full matrices, answer-paired effects, behavior strata, and label-task alignment.

Hallucination detection for retrieval-augmented generation (RAG) is usually evaluated on natural-language document evidence. However, grounded generation systems increasingly rely on structured inputs: source code, developer-tool output, markdown documents, tables, and repository metadata. We introduce a unified benchmark for span-level hallucination detection over code, tool output, structured documents, and existing natural-language RAG datasets. The benchmark is built by starting from grounded correct answers, injecting localized hallucinations with exact character labels, and validating the code test split with evidence-based review. Our fine-tuned Qwen3.5-2B detector reaches 0.689 span-F1 on the unified test set and 0.60 on the code-agent source, where it substantially outperforms LettuceDetect-large (0.17) and the strongest zero-shot LLM judges we evaluated (at most 0.22). The same model remains competitive on established natural-language benchmarks, with 81.8 RAGTruth example-F1 and 0.724 English PsiloQA IoU.

AI-generated text now constitutes a substantial and growing fraction of indexed web content, creating a feedback loop in which AI assistants may retrieve and condition on their own prior generations. Prior work shows that recursive training on model generations causes model collapse: a loss of output diversity and distributional drift. We ask whether a similar collapse emerges at inference time, when LLMs retrieve their own generations via RAG without any parameter updates. We measure collapse as the progressive convergence of responses to a narrow set across recursive retrieval rounds. Across three contamination simulations and four open-source LLMs, answer collapse is rapid and consistent under direct contamination, but slower, partial, and model-dependent under realistic competitive retrieval. Collapse can also degrade downstream accuracy on factual multi-hop QA, though the effect is inconsistent across models. We further show that the self-generated documents we introduce are cited far above their share of the retrieved context, an effect that is not fully explained by document quality, writing style, AI authorship, and self-preference. Finally, we propose and evaluate mitigation strategies at both the retrieval and generation stages, none of which reliably prevent collapse. As AI-generated content continues to populate the open web, collapse-resistant retrieval design becomes a prerequisite for trustworthy AI assistants.

Search APIs expose ranked snippets, URLs, and metadata on which agents decide whether to answer, search again, or fetch pages. We evaluate these interfaces as decision surfaces on a fixed sample of 100 questions from the 254-question SEALQA-HARD subset, using one frozen GPT-5.4 agent, a fixed orchestration harness, and a shared page-fetch backend across Brave, Tavily, and Firecrawl. A Kimi-K2.6 oracle labels visible URL-level evidence;a separate answer audit yields 25, 25, and 26 correct answers out of 100. These counts indicate similar observed accuracy but do not establish equivalence. Under the tested configurations, Brave exposes more pre-fetch support alongside a larger snippet surface; Tavily has a larger rank-1 share among trajectory-pooled supporting observations; and Firecrawl is associated with broader exploration. First-search query-level metrics distinguish support availability from ranking, while contradiction exposure complements contradiction-to-gold ratios. A retrospective oracle union covers 44/100 questions, versus 26/100 for the best individual provider: 18 percentage points of headroom. The observed evidence and action differences motivate evaluating search APIs jointly with agent policy and retrieval budget.

Small Language Models (SLMs) under 10B parameters are increasingly deployed on edge devices for privacy-sensitive, latency-critical applications, where their factual reliability is pivotal and no larger model is available to fall back on. These SLMs frequently exhibit sycophancy: they abandon a correct answer the moment a user confidently asserts an incorrect one. In this paper, we study where this failure originates and how to fix it without retraining. We introduce a prefix/suffix multiple-choice construction that reduces every question to a single answer token, enabling layer-wise analysis of exactly when a model commits to the truth or the lie. Using this probe, we show that sycophancy is pervasive and, crucially, that it is a routing failure rather than a knowledge failure, i.e. middle layers retrieve the correct answer with a strong, linearly separable signal but the final alignment layers route the residual stream away from it. We also show that this behaviour is seen far more in procedural reasoning, where the SLM has to think in a step-by-step fashion, than in factual recall, where the answer depends heavily on pretraining information. We quantify this with a layer-wise Criticality Score and localize the suppression to a narrow band of late layers. Building on this, we propose Latent Truth Decoding (LTD), a training-free intervention that decodes from the model’s epistemic peak rather than its final layer.

Automatic metrics stand in for native-speaker evaluation in low-resource retrieval-augmented generation (RAG), so their validity determines what can be concluded about these systems. We test whether common metrics detect re- trieval and grounding failure in one fixed RAG pipeline across ten Indic languages. They do not merely miss the failure in Santali; they invert it. Hit@5 is 0.011, yet raw XLM-R BERTScore is the highest in the set at 0.902, changes little when retrieved evidence is re- moved, and assigns nearly the same score to matched and unrelated references. The cause is coverage: the scorer maps 39% of Santali tokens to <unk>, and a scorer whose vocab- ulary includes Ol Chiki restores discrimina- tion while LaBSE does not. Retrieval fails for the same reason. The gold article is indexed for 80% of Santali queries, but the retriever shares XLM-R’s tokenizer and collapses before reranking. Judge-based faithfulness and con- text precision, computed by a model with a dif- ferent tokenizer, flag the failure independently. A metric that cannot represent a script does not become noisy; it becomes constant and high. We argue that metric validity in low-resource languages must be established per language, and that evidence removal, reference shuffling, and the scorer’s <unk> rate can be important additions.

Modern language models are typically deployed using all transformer layers during inference, implicitly assuming that every layer contributes positively to downstream performance. Prior studies on layer skipping often focus on limited settings and commonly associate removable layers with specific transformer regions, particularly middle layers. However, whether single-layer skipping generalizes across diverse tasks, languages, and model architectures remains underexplored.

In this work, we conduct a multilingual and multi-task empirical study of single-layer skipping in modern instruct-tuned language models. We evaluate 9 text-only models spanning multiple model families and parameter scales, including Gemma 3, Llama 3.2, and Qwen 3, OLMo 2 without modifying model parameters. Our experiments systematically evaluate single-layer skipping across all layers on diverse benchmarks covering NLU, reasoning, bias evaluation, code generation, instruction following, and multilingual knowledge evaluation, including MMLU across 57 subject domains and 10 typologically diverse languages.

Our results show that the effect of single-layer skipping is highly context-dependent. We show that skipping a single transformer layer is an effective strategy for leveraging the capabilities of a language model: there is a high chance of achieving better benchmark performance while simultaneously reducing computation during inference. However, the optimal skipped layer varies depending on model architecture, task, domain, and language. Beneficial skipped layers occur most frequently in the second half of the network, though they also emerge in early layers in some cases. We further observe that performance improvements are more common in structured prediction tasks, whereas degradations more frequently occur in tasks requiring precise generation.

Named Entity Recognition for dialectal Arabic remains severely constrained by data scarcity, and LLM-based augmentation is the natural remedy. We study LLM-based augmentation for the Lebanese and Palestinian sub-corpora of Wojood 2024, using LLaMA-3.1-8B-Instruct as a generation backbone, and compare three strategies (entity-level, context-level, combined) across two k-shot sampling paradigms. The proposed multi-dimensional pipeline yields a per-sentence composite score Q(s) that exposes a sharp dialectal-authenticity gap between entity- and context-level rewriting (DA 0.92 vs. 0.51). Despite lower register fidelity, context augmentation with curriculum training delivers the most consistent gains (Curras F1 +5.70 pp); entity augmentation lifts Lebanese F1 from a zero baseline to 50.00 on Lebanese-only training, a figure the 85-sentence test set makes coarse. Ablation, weight-sensitivity, threshold sweep, a random-selection control, and a second LLM backbone show that the dialectal-authenticity dimension carries most of Q(s)’s protective effect on Lebanese and that generator choice is itself a first-class register-preservation lever.

We compare diffusion-based and autoregressive language models under CheckList-style perturbations. We frame semantics-preserving perturbations as potentially containing an off-manifold component and hypothesise that diffusion denoising can reduce the resulting output change relative to autoregressive decoding. We evaluate eight instruction-tuned 7-9B models (three diffusion, five autoregressive) across sentiment, QQP, and SQuAD using behavioural pass rates and a perturbation coefficient that measures output change relative to input change. Results are task-dependent. Behaviourally, diffusion gains 18.2 and 11.0 points overall on QQP and SQuAD, while autoregressive models perform better on the sentiment and SQuAD invariance slices. On QQP, the all-model coefficient ratio does not favour diffusion ( ̂γ_AR/ ̂α_Diff ≈ 0.91); excluding two autoregressive models with strong label-prior effects as a diagnostic sensitivity analysis yields 1.23 overall and 1.48 on invariance tests. Diffusion also performs worse on perturbations that should change the output. Thus, the observed stability differences are task-conditional, and because the empirical diffusion coefficients exceed one, the experiments do not directly validate the contraction assumption of our theoretical proposition. Data and code are available at: https://github.com/AarushRathore1729/dlm_ar_final.

Large language models can hallucinate with the same fluent confidence they bring to correct answers, making reliable hallucination detection essential for trustworthy deployment. Existing detectors based on LLM hidden states often rely on cosine or linear readouts that ignore the anisotropic structure of the representation space. We introduce DTS (Discriminant Truthfulness Scoring), a lightweight, gradient-free detector that whitens hidden activations using within-class covariance and scores responses with a closed-form Fisher discriminant. Across Qwen-2.5-7B and Llama-3.1-8B on four QA datasets, DTS ranks first in five of eight settings and second in the remaining three. Whitening improves over cosine scoring by 7.8 AUROC points on average, identifying the readout as a major bottleneck. When truthful and hallucinated representations overlap, class-mean steering provides additional gains without learning a steering vector. DTS requires only a single forward pass and closed-form computation, making it substantially cheaper than sampling- and gradient-based alternatives, while also extending effectively to long-form generation.

Grounded language models are usually evaluated by adding relevant context, but multi-turn dialogue also contains unsupported user claims that may contaminate later factual answers. We study post-pressure recoverability: whether a model returns to clean-context behavior after a user repeatedly advocates a wrong answer and then withdraws that pressure. We introduce a recovery-after-pressure protocol for multiple-choice factual dialogue and measure sycophancy hysteresis, the residual probability assigned to the user-advocated wrong answer relative to a clean-context counterfactual. Across seven instruction-tuned open-weight models and two factual benchmarks, ordinary reset often reduces but does not erase pressure-induced bias. History-preserving repairs such as user retraction, system reset, and self-verification recover only 2–3/14 model–dataset pairs under the strict clean-restoration diagnostic, whereas operations that change the effective context are substantially more reliable; the two conditions that remove the pressure-bearing history entirely, fresh-context deletion and context truncation, recover 14/14. In an oracle trusted-evidence condition across fourteen model–dataset pairs, preserving the pressure-bearing history while adding benchmark-derived trusted evidence increases accuracy from 0.368 to 0.929, while wrong-answer following falls from 41.2% to 4.3%. Controls show that the effect is not explained by dialogue length, repeated confidence, plausible distractors, mere false-answer mention, or option-label inertia. These results suggest that faithful grounded dialogue requires evaluating which prior context should be treated as evidence and which should be removed or quarantined before answering.

A single faithfulness score can hide failures that require different repairs: a RAG answer may be grounded in text whose source is stale, incomplete, contradicted, low-authority, or insufficient. We introduce ContextTrace, a deterministic, local-first SDK and CLI for evidence-chain forensics that separates claim support, citation state, source condition, abstention need, root cause, repair, and regression prevention. We also introduce Groundedness-Truth-Gap, a suite for controlled conformance, while frozen RAGTruth and stress tests serve as failure-safety controls. ContextTrace’s central contribution is an inspectable evidence-chain contract. On a fresh 84-case public-document evaluation, it catches 76 of 77 failure-bearing cases and preserves all seven clean cases (balanced accuracy 0.994; MCC 0.929), the same binary confusion counts as a frozen GPT-4.1-mini judge, while producing typed diagnostic records without additional model calls. The clean-case sample is small, and fine-grained label, root, and unverifiable attribution remains limited. Post-review counterfactuals further show strong dependence on source metadata and temporal cues. We release a blinded 50-case annotation packet and scorer; independent adjudication remains pending. Our claim is reproducible fail-closed diagnosis, not truth verification or broad evaluator superiority.

Frontier multimodal large language models (MLLMs) have achieved strong performance in visual question answering (VQA), OCR-intensive image understanding, and multimodal reasoning. However, they can still produce linguistically plausible answers that are not supported by visual evidence, particularly under ambiguous OCR, weak visual evidence, and challenging spatial or physical reasoning. We introduce GroundVQA, an adversarial VQA benchmark designed to evaluate visual grounding, hallucination resistance, OCR robustness, uncertainty handling, and spatial and physical reasoning. GroundVQA contains 3,877 manually reviewed question–answer pairs constructed from approximately 2,400 public-source images and around 100 self-collected real-world images. We evaluate representative frontier MLLMs on a stratified subset of GroundVQA and complement the quantitative evaluation with qualitative analyses of recurring model strengths and failure modes. Our findings highlight a persistent reliance on semantic priors when visual evidence is ambiguous or insufficient, motivating evaluation frameworks that emphasize grounded reasoning, calibrated uncertainty, and adversarial premise rejection beyond conventional VQA accuracy.

Recent advances in Multilingual Large Language Models (MLLMs) have improved cross-lingual conversational capabilities, yet generating reliable, institutionally usable text in low-resource languages like Bangla remains a critical bottleneck, particularly for compact, low-resource LLMs. Beyond surface fluency, state-of-the-art and small open-weight models frequently exhibit deeper generation failures in Bangla: incoherent or off-topic output, structural misformatting, and inconsistent honorific registers, that render text unusable in formal contexts. To address this, we introduce BLADE (BangLa Application and DialoguE generation), a culturally aligned instruction-tuning dataset and benchmarking framework comprising 4,196 curated interaction pairs. We use this resource to fine-tune and evaluate compact open-weight architectures, including DeepSeek-8B, LLaMA-3.2-3B, Qwen2, and TigerLLM-1B, via LoRA adapters under 4-bit NormalFloat (NF4) quantization on limited hardware. Our evaluations show that targeted fine-tuning on BLADE yields substantial improvements in generation reliability, structural fidelity, and register consistency even for the smallest models, establishing a rigorous benchmark for closing generation and pragmatic gaps in low-resource multilingual text generation. Code and dataset: https://github.com/ashuvo25/Bangla_Application_LLM

Literature-grounded question answering requires a system to retrieve the right paper, locate the right evidence within it, and generate an answer faithful to that evidence. These are three separate stages, each with its own failure mode. We present a stage-wise diagnostic study of this pipeline on LitTraceQA, a shared task requiring paper retrieval, coarse evidence localization (table/figure/page), and structured answer generation. Our pipeline combines cross-model hypothesis diversity for retrieval, a filter-then-rank evidence localizer (caption-pattern match, embedding similarity, evidence-type gate), and cross-model semantic entropy for generation-time answer aggregation and confidence estimation. We use an oracle-substitution ablation ladder, sequentially handing the pipeline the gold paper and then the gold evidence value, and reading off the resulting change in each stage’s own metric. These are sequential oracle-intervention gains, not an additive error decomposition: the stages are measured with different quantities and are not independent. Read that way, the ladder points at evidence localization as the dominant remediable bottleneck — multiple-choice accuracy rises from 0.42 with predicted evidence to 0.76 with the gold evidence value, and end-to-end accuracy stays near 0.29 even on the questions where retrieval already returned every gold paper. We also test whether cheap uncertainty signals computed at retrieval time predict which stage will fail before it does. Along the way we surface two infrastructure-level findings with direct implications for anyone building literature-grounded QA systems. First, evidence page numbers in this and similar benchmarks are keyed to a paper’s canonical published version rather than its arXiv preprint; a real failure mode when a paper has been substantively revised, though a systematic paired ablation across 45 questions finds it is an occasional rather than pervasive hazard in this pool. Second, a substantial fraction of papers hosted on OpenReview (ICLR/ICML/NeurIPS) are unreachable by ordinary scripted HTTP requests due to a JS bot-challenge, a venue-correlated failure mode unrelated to any modeling choice. We report our diagnostic protocol, quantitative results with bootstrap confidence intervals given the benchmark’s small public dev set, and a qualitative failure taxonomy, framing this as a resource and lessons-learned contribution rather than a leaderboard claim. All multiple-choice numbers are a best-case diagnostic condition: they assume access to the gold option text, which a genuine blind submission would not have.

Large language models (LLMs) are increasingly used to generate scientific reports, but they can produce references that appear plausible while containing corrupted metadata or pointing to papers that do not exist. We introduce CITECHECK, a hybrid framework for citation hallucination detection that verifies whether a citation corresponds to a real scholarly work and whether its metadata is faithful to that work. CITECHECK retrieves candidate publications from external scholarly sources, compares the citation against the retrieved candidate using a structured LLM verifier, and maps verifier scores into three labels: Exact, Minor, and Major. We also construct a 982-citation physics benchmark with controlled corruptions that capture both subtle metadata drift and fully fabricated references. On the held-out test set, CITECHECK achieves 88.7 macro-F1 and 88.9% accuracy, outperforming GPT, Claude, and Gemini baselines, including web-search and few-shot variants. These results show that reliable citation verification benefits from combining scholarly retrieval, structured LLM-based comparison, and calibrated decision rules.

Policy compliance in enterprise conversational AI is not binary: the same response can satisfy one stakeholder’s policy while violating another’s. Existing evaluations largely treat compliance as a single-policy problem, which makes cross-role conflict cases difficult to construct and assess. GRACE (Generating Conflict-Aware Cases for Evaluating Policy Compliance) addresses this gap by generating matched normal and conflict cases from the same underlying scenario, assigning role-wise expected verdicts through independent per-role policy evaluation, and scoring responses independently for each role rather than collapsing compliance into a single binary label. Across four domains, this process yields 1,109 labeled scenarios, including 309 cross-role conflicts. Evaluating five LLMs shows that deny cases are easiest, allow cases are depressed by over-refusal, and conflict cases are consistently hardest, revealing failures that single-label evaluation obscures. Per-persona analysis further shows that these errors are localized to specific domain-role combinations rather than uniform across models. GRACE therefore supports both policy review and deployment-time model selection.

Adaptive Bitrate (ABR) decision-making is essential for optimizing Quality of Experience (QoE) in video streaming. Traditional ABR methods typically lack fine-grained content understanding and face challenges in balancing multi-dimensional QoE objectives under real-world edge device constraints. To overcome these limitations, we propose **StreamSense Agent**, a novel framework that reformulates ABR as a sequential decision-making process informed by content and network awareness. Our approach introduces three key innovations: (1) a Dynamic Content-Aware Entity Graph (D-CAEG) for structured video representation, (2) a composite QoE-driven optimization objective integrating perceptual quality, rebuffering, smoothness, and computational cost, and (3) a hierarchical resource-aware architecture enabling real-time control through an efficient lightweight actor. We further adopt an offline-to-online co-adaptation training paradigm with federated learning for continuous adaptation. Since effective ABR control requires the same core competencies as general sequential decision-making—multi-step planning across future video chunks, dynamic information gathering from heterogeneous network signals, and adaptive strategy adjustment under resource constraints—we validate these transferable capabilities on five established benchmarks. Specifically, multi-hop search tasks (BrowseComp, BrowseComp-ZH) mirror the integration of dispersed network-trace and content-complexity signals into a bitrate plan; complex navigation tasks (WebWalkerQA) correspond to sequential buffer-state traversal under volatile bandwidth; and information-extraction tasks (Xbench-DS, SimpleQA) reflect the precision required when reading real-time streaming telemetry. StreamSense Agent achieves state-of-the-art results, attaining accuracy of 16.7% on BrowseComp, 30.5% on BrowseComp-ZH, 42.1% on Xbench-DS, and 54.2% on WebWalkerQA. Deployment-oriented evaluations further confirm its computational efficiency, achieving an inference throughput of 12.5 requests per second with a memory footprint of only 2,100 MB under edge-like constraints, yielding a normalized QoE score of 0.92. Comprehensive ablation studies validate the necessity of each proposed component. This work establishes a new paradigm for intelligent, edge-deployable bitrate adaptation.

Pairwise LLM-as-judge evaluation should preserve response identity when presentation order changes. Across four judge models, short A/B prompts yield 70–100% second-position selection (72/80 comparisons overall). Targeted prompt ablations show format sensitivity: label changes alter parsed choices, and a neutral buffer produces complete first-position selection on 59 fixed-order comparisons, reversing rather than removing position preference. We use a position-swapped evaluation protocol, retaining a directional verdict only when both orders select the same response. In a new 16-condition GPT-4.1 factorial audit, analysis-first and a verdict prefix raise directional coverage by 5.6 and 6.5 percentage points, respectively. On 40 Arena-Hard queries with three generators, the 120 pairs comprise 62 directional results, no double ties, 14 swap failures, and 44 invalid pairs. Cross-Model Agreement (CMA) yields 25 unique query winners. A retrospective domain-routing check predicts 17 of these 25 labels using rules fitted outside each test fold. These labels and any oracle score remain judge-relative; human-preference validity and downstream training gains are not established.

Query rewriting is now a common preprocessing step in retrieval-augmented generation (RAG) systems, yet it can silently damage grounding before generation begins: a fluent rewrite may drop the exact terms that retrieve the evidence. We study this retrieval-interface failure mode across five benchmarks, three retriever families, three rewriting strategies, and two open-weight LLMs. In the main Zephyr-7B setting, unconstrained rewriting degrades nDCG@10 by more than 0.05 for 26–76% of queries, with harmful cases often exhibiting evidence-anchor drift: rare domain terms, named entities, and numerics are replaced by less discriminative paraphrases. We propose a training-free constraint layer that identifies candidate evidence anchors, prompts the LLM to preserve them, verifies preservation deterministically, and falls back to the original query when verification fails. The method requires no retriever training, no logit access, and no constrained decoder. It reduces mean qrel-observed harm from 48% to 10% on BEIR and from 26% to 6% on Natural Questions, with the same qualitative pattern across tested retrievers, rewriting strategies, and 7B open-weight models. We isolate benchmark-labeled evidence exposure as a pre-generation condition for downstream attribution rather than evaluating generated answers. The results support a practical design principle: LLMs may expand retrieval queries, but the retrieval interface should explicitly protect the lexical anchors on which evidence access depends.

Creative search platforms help users turn short, vague queries into searches and generation prompts. To be useful, a suggestion has to fit the context it appears in, the asset type, the season, and the user’s region, and it has to be grounded in assets the platform can actually retrieve. A fluent suggestion that returns nothing wastes the user’s attention and erodes trust. Building such a system is hard because there is no ready-made training data: user logs are sparse, noisy, and privacy-constrained, and they say nothing about context or grounding. We first solve this with corpus-grounded synthetic supervision. A two-stage offline pipeline turns 59M asset captions into context-aware training data, constrained by a Creative Knowledge Graph and grounded through retrieval, and fine-tunes a single low-latency model that serves related suggestions, refinements, and generation-ready prompts in one retrieval-free step. We check grounding by running suggestions against the production index: offline, retrieval grounding raises the share of well-grounded suggestions by over 30% relative, and in production about 99% of served suggestions return real assets (a null rate near 1%). The deployed system is preferred over the prior log-based suggester in human evaluation and improved the primary business metric in production. As the system reached new surfaces, two requirements emerged that the supervised data met poorly: single-word suggestions for space-constrained interfaces, and selective seasonal and regional context—added in the harder cases the deployed model missed, and suppressed where it does not apply. Rather than regenerate the data and retrain, we add a Group Relative Policy Optimization (GRPO) stage whose hybrid reward pairs an LLM judge with programmatic checks for format, length, and diversity. It improves seasonal alignment by 18.6 and regional alignment by 5.2 points (on a 0 to 100 judge scale) and raises single-word compliance from 15.1% to 99.1%, a format absent from the supervised data, while keeping relevance and diversity intact and leaving the retrieval-free serving path unchanged.

LLM-as-a-Judge is widely used to rank model outputs and populate leaderboards, yet its run-to-run reliability is under-characterized. We study repeated identical evaluations across six judges from three providers spanning three model generations (2024-2026), with 50 pairwise and 50 pointwise trials per question on 29 tasks. Three findings stand out. First, pairwise verdicts are unstable, with an 11-14% mean flip rate on identical re-runs, and this has not improved across generations: the newest, cheapest deployable judges flip just as often as 2024 models. Second, a dedicated swap-order experiment (8,700 judgments) shows the fixed-order asymmetry is order bias, not self-preference, and that order bias has not been fixed either: the 2026 judges reverse its direction, favouring the second slot where earlier judges favoured the first, so a first-position check records them as unbiased while they still decide 21-41% of matched comparisons on slot rather than content. Third, forced-choice verdicts and scalar scores come apart, and ~11 repeated trials are needed to stabilize a verdict. We argue that multi-trial aggregation, symmetric order randomization, and uncertainty reporting should be standard practice.

Context compression reduces generator input in retrieval-augmented generation, but answer quality alone does not characterize citation attribution. We measure citation attribution across compression methods and budgets, comparing reranking, extractive selection, abstractive summarization, token pruning, and an extract-cluster-rewrite construction on ASQA and QASPER under a fixed generator and primary entailment evaluator. On ASQA at a nominal 0.25 budget (achieved compression 0.08), a RECOMP-style compressor’s citations score 0.86 precision against its summaries but 0.12 against source spans under our re-attributability protocol. These estimates depend on a shared NLI model for span recovery and citation scoring and lack independent human calibration. Extractive selection’s observed grounded precision ranges from 0.43 to 0.49 across nominal budgets from one-half to one-tenth of the ASQA context, while answer quality declines. For the same RECOMP setting, claim verification after source recovery yields an unsupported rate of 0.88 versus 0.17 when checking summaries. This gap persists beyond structural rejection of missing provenance, but remains evaluator-dependent. A 200-question TRUE T5-XXL audit also finds emitted–grounded gaps under both fixed and recomputed source mappings, without establishing human-calibrated support rates.

Single-GPU deployment of 70B-parameter language models on an NVIDIA GPU is constrained by device memory, long-context throughput, and engineering integration cost. We cast single-GPU inference as a budget-aware design problem over these three axes and study how pruning, quantization, and KV-cache compression interact under realistic execution. Controlled ablations show that layer-wise pruning makes weight quantization more robust. KV-cache sparsification complements INT8 KV quantization by reducing memory without hurting decoding speed, while static vector quantizers often conflict with dynamic caching. Guided by these coupling results and explicit budget tracking, we assembled a practical pipeline and compressed a 70B model to about 33 GB, sustained about 57 tokens/s on 10k token prompts on a single A40, and kept absolute accuracy within 5% on common and reasoning benchmarks. We contribute design rules and a reproducible evaluation protocol that jointly report quality, memory, and end-to-end speed, and we provide a foundation for automated pipeline search under realistic single-GPU constraints.

Hallucination—where a language model generates outputs that are factually incorrect or unsupported by the source—is a major challenge for both prompted and fine-tuned language models. Detecting hallucinations is difficult due to the opaque reasoning processes of LLMs, which often provide little insight into why a model’s output may be inaccurate.

In this work, we investigate whether an LLM can use an alternative, low level, symbolic competence such as SQL for unsupervised hallucination detection in some high level task. For this, we make an LLM build an SQL database from reference documents. This SQL database is then used for reasoning over the reference and the sampled response in a hallucination detection pipeline that is grounded in the database, thereby providing a neurosymbolic checkup.

On RAGTruth and DiaHalu hallucination detection datasets, we find that our approach improves on direct prediction and competes with state-of-the-art hallucination detection methods, while not requiring domain-specific fine-tuning. Instead it relies on a low-level general competence already present in LLMs. This warrants further investigation of low-level LLM competences in neurosymbolic approaches.

In modern RAG pipelines, query augmentation methods such as HyDE and query expansion are applied to every query, resulting in substantial LLM inference costs and increased end-to-end latency. The empirical justification for this overhead in real production traffic remains largely unexplored. We present a case study of the Danish National Encyclopedia, evaluating five retrieval workflows over 20,000 query–workflow pairs from production traffic and synthetic conditions. In this system, synthetic queries suggest that LLM augmentation is needed for over 90% of queries to achieve high retrieval coverage. However, under our production deferral policy, only 27.8% of real user queries need LLM augmentation. We call this gap the Coverage Illusion and attribute it to a structural mismatch between synthetic and real query distributions. Pre-retrieval routing cannot resolve this gap, as the need for LLM augmentation is only revealed after searching the index, a result confirmed by our evaluation of four machine learning paradigms. The coverage gap, undetectable from the query alone, motivates a post-retrieval cascade that runs workflows in cheapest-first order and escalates to LLM augmentation only when a step returns no documents. Operating entirely without training overhead or secondary serving infrastructure, the cascade improves quality by +0.140 Composite Overall points over Always-HyDE, reduces latency by 31.8%, and serves 72.2% of real user queries without LLM augmentation.

Language models often receive a question together with a claim about what another source answered. We audit whether such claims destabilize answers in multiple-choice question answering. For each item, we hold one wrong option fixed across misleading conditions and vary the cue template attached to it. We introduce neutral-conditioned misleading cue adoption rate (NC-MCAR), which measures switches to that option only on valid cued trials where the same model first selected the gold answer under a neutral prompt. This is a measure of answer instability, not proof that the model knew the answer or that all deference is irrational. We evaluate four instruction-following models on MMLU-Pro and IndicMMLU-Pro in English, Hindi, Bengali, Tamil, and Telugu. Across 220,000 outputs, the expert template yields 41.1% aggregate NC-MCAR, compared with 12.5% for the majority template. These two conditions use the same wrong option and final instruction. Filler accuracy remains well above expert-wrong accuracy, while correct-cue prompts have high valid-response accuracy. The audit documents answer instability relevant to grounding under the tested forced-choice prompts: a bare, unverified source claim can outweigh an answer that was previously consistent with the task evidence.

Grounded language-model pipelines can be divided into three stages: selecting an object, retrieving passages for it, and using that evidence to answer. If the selected object must reach the reader, losing it breaks the handoff. Benchmark recall checks the dataset-linked object, which can differ. We audit 600 HybridQA questions across three selector families. On 1,463 resolvable records where the selected object matches the dataset-traced passage, exact key lookup and exact title matching return the object every time. With every ranked rule given the same decoded selected title, body-only BM25 omits it on 389 records (26.6%) at cutoff five, while hybrid retrieval with reranking omits it on 14 (1.0%). The two identities differ on 329 of 1,792 resolvable records. With original-question rankings, their top-five checks disagree on 106 records (5.9%). Frozen reader comparisons associate the aligned object’s presence with 28.6 to 31.0 points higher exact match. In a deliberately selected 64-item cohort, removing that passage sharply lowers exact match, while removing a similar-length comparison passage does not reproduce the drop. We release the Returned-Object Profile (ROP), an executable record of the target, returned-ID field, cutoff, membership rule, and complete expected population, with data and an offline replay.

Effective task-oriented dialogue (TOD) requires agents to coordinate two tightly coupled decisions of when to invoke external tools and how to formulate a response that satisfies the user’s goal. However, their turn-level behavior remains weakly grounded under fixed zero-shot prompts. Meanwhile, existing retrieval-based TOD agents primarily retrieve task-relevant content from external knowledge sources, guiding which information to include but offering limited guidance on how to select tools and formulate responses in similar dialogue contexts. To mitigate this, we introduce TRACE, a Turn-level Retrieval Agent with Contextual Experiences for TOD [<https://github.com/aistairc/TRACE-TOD>]. Instead of relying on retrieved factual records alone, TRACE retrieves goal-conditioned dialogue snippets as behavioral demonstrations at each turn. These snippets capture similar user goals, constraints, tool-use decisions, and response patterns to guide both tool calls and response generation. Experiments on MultiWOZ 2.1 and Schema-Guided Dialogue (SGD) show that TRACE improves Inform and Success over a competitive LLM agent and end-to-end fine-tuned models under both simulation-based and static-user evaluations. Human evaluation and retrieval-focused analyses further show that goal-conditioned snippets and retrieval quality drive these gains.

Retrieval-Augmented Generation (RAG) is widely used for knowledge-intensive question answering (QA) but typically relies on large language models (LLMs) as generators, which raises cost and privacy concerns. We propose TripletRAG, a simple, training-free method that converts LLM-teacher behavior into triplets (synthetic question, retrieved passages, reference answer), which are then provided as exemplar context to a small language model (SLM) at inference. TripletRAG requires only offline synthetic QA generation and retrieval; no SLM finetuning is needed. Across SQuAD and Natural Questions (KILT) datasets, triplet guidance substantially increases judge-based accuracy, and improves groundedness on SQuAD for models in the 1B–3B parameter range, with negligible or inconsistent gains for larger models (7B+). Compared to a strong lightweight baseline (MiniRAG), TripletRAG gives markedly better groundedness and correctness while keeping the pipeline simple and practical.

Diagnosing failure patterns in Deep Research Agents (DRAs) remains a critical challenge. Existing benchmarks predominantly rely on end-to-end evaluation, obscuring intermediate hallucinations that accumulate throughout the research trajectory. To bridge this gap, we propose a shift from outcome-based to **process-aware evaluation** by auditing hallucinations in the full *plan-search-summarize* trajectory. We introduce the **PING Taxonomy**, which categorizes DRA hallucinations into four complementary types: **P**ropagation, **I**ntent, **N**oise-induced, and **G**rounding. We further instantiate this taxonomy into a fine-grained evaluation framework that decomposes trajectories into atomic actions, claims, and sub-queries for rigorous verification, and we validate its reliability on standard fact-checking benchmarks and human-reviewed trajectories. Leveraging this framework to isolate 100 hallucination-prone tasks, including adversarial scenarios, we curate **DeepHalluBench**. Experiments on six representative DRAs show that, on our hallucination-prone stress-test set, all evaluated systems still exhibit non-negligible reliability gaps. Furthermore, our diagnostic analysis traces these failures to systemic deficits, especially hallucination propagation and cognitive biases, providing actionable insights for future architectural optimization. Code and data are available at https://github.com/yuhao-zhan/DeepHalluBench.

We introduce Conversational Object Grounding, a new task in which an agent must identify a target object in a complex 3D scene by engaging in natural language dialogue with a user who knows the target. It extends multimodal reference resolution on a single referring expression to conversations over multiple turns, combining it with clarification question generation and answer understanding to resolve ambiguities. For this task, we propose a zero-shot modular approach that combines the reference resolution, object captioning, and dialogue capabilities of foundation models, using prompt chaining in conjunction with an image-grounded scene graph. Evaluations on a new benchmark constructed with 3D scenes from ScanNet show that the proposed modular approach outperforms end-to-end baselines in terms of both grounding performance (up to 13% improvement) and image processing cost (up to 63% reduction).

Retrieval-Augmented Generation (RAG) systems are increasingly adopted in medical question answering, where generated answers depend heavily on retrieved biomedical evidence. However, demographic perturbations in clinical queries can substantially alter retrieval behavior, causing retrievers to surface demographically correlated yet clinically irrelevant evidence. We refer to this phenomenon as retrieval shift, where demographic-sensitive retrieval representations propagate bias into downstream generation. To investigate this issue, we construct a demographically perturbed MedQA dataset and systematically analyze retrieval instability under demographic perturbations across representative medical RAG systems. Our analysis shows that RAG systems are substantially more sensitive to demographic perturbations than standalone LLMs, indicating that retrieval plays a central role in propagating demographic bias. To mitigate retrieval shift, we propose Bias-Aware Neutral Debiasing (BAND), a rotation-based retrieval debiasing framework that suppresses demographic-sensitive retrieval directions while maintaining clinically relevant retrieval behavior. Experiments demonstrate that our method consistently reduces retrieval shift while largely preserving downstream task performance across multiple medical RAG systems. Our code and dataset are available at: https://github.com/hrimlee/BAND

LLM coding agents such as GitHub Copilot, Cursor, and Claude Code now author a large share of the code that teams merge, and they are active in an estimated 22% of public GitHub projects. The output is fast and usually correct in the small. The trouble is that an agent optimizes for the task in front of it, not the global shape of the system. That cost is now measurable. Agent adoption causally raises code complexity by about 42% and static-analysis warnings by about 30%, and the most common reason an agent’s patch fails is a shallow grasp of the surrounding architecture. We call this widening gap between a system’s intended and realized architecture architectural drift. The obvious remedy, stating the rules in the prompt, does not hold at repository scale; the evidence is that only deterministic, executable enforcement conforms reliably. We make two contributions. The first is a taxonomy of LLM-induced architectural drift: five deterministically enforceable families plus one open frontier, each tied to the architectural primitive it corrupts and the model behavior that produces it. The second is GyroCompass, a deployment-oriented framework. It extracts a living architecture model from source, serves that model to the agent as planning-time context (prevention), and enforces it through a deterministic gate that, run in CI, cannot be bypassed (detection). On a labeled benchmark of 208 cases (104 violations, 104 realistic decoys) spanning Python and JavaScript/TypeScript, GyroCompass reaches 88.5% recall at a 5.8% false-positive rate, versus 57.7% recall at 42.3% false positives for a naive pattern baseline (McNemar p<0.001); on the original 54-case core it separates violations from look-alikes at 100% recall and zero false positives. Head-to-head on matched subsets, a five-tool comparison (Semgrep, Bandit, CodeQL, pip-audit, import-linter) reaches at most 65% recall. Extraction runs at over 800 files per second, cheap enough to apply on every commit, and its output matches independently constructed references at 0.95-1.0 component precision/recall across four repositories. In an added-line analysis of 1,779 commits of a 10k-star production repository whose maintainers document their layering rules for AI agents, violations of the documented intent continued to land, including after an agent-facing AGENTS.md stated the rules,which is the enforcement gap GyroCompass closes. On the prevention side, serving the architecture to a coding agent as context shifts its layering behavior, but the effect is model-dependent across three model families, which is itself why the deterministic gate, not prompt context, is the binding guarantee. We discuss the prevention/detection trade-off, lessons from instrumenting agent workflows, and what reliable guardrails ask of a team in practice. GyroCompass is a working system intended for real-world deployment; benchmark, harness, and analysis scripts are released.

Post-training with internal feedback can improve a targeted knowledge-grounded behavior while silently degrading complementary abstraction and logical-consistency capabilities. We propose a conjunctive evaluation protocol that accepts a method only when a designated target improves and complementary probes remain non-inferior (with a stricter secondary criterion requiring complementary equivalence), while also reporting retention and predictive drift. We instantiate the protocol in DION-Bench, combining two lexical/ontological probes (same-category exceptions; abstraction recovery) with a synthetic SAT-checked logical-consistency probe. On Qwen3-8B-Base, a task-specific self-judged DPO adaptation improves OntoNeg-WN by 5.27 percentage points but reduces ConceptShift by 37.50 points—despite humans reaching 82.7% and frontier LLMs 44–73% on OntoNeg—whereas mean change across five external accuracy suites is only −1.70 points. A development-tuned decoding rule partially mitigates ConceptShift regression to 12.93/14.60 points on the original/holdout splits while retaining most of the target gain. These results show that faithful post-training claims require joint reporting of gain, complementary abstraction regression, retention, drift, and equivalence—not proof that a single shared knowledge item degraded.

Slide narration generation can introduce unsupported visual or decorative claims. We describe a pipeline that indexes slide content with deterministic identifiers, generates an internally cited template, and applies heuristic checks before and after an optional language-model rewrite. A narration-stage study covers 30 publicly sourced presentations (279 slides). The same decks and procedure were used, but stochastic vision evidence was regenerated by condition. In one run per condition, the complete configuration produced 5/279 proxy flags (1.79%) versus 22/279 (7.89%) for A1, a 77.3% relative difference (slide-paired exact p ≈ 2.2 × 10⁻⁴; deck-blocked p = 0.0156); B produced 20/279 (7.17%). Saved labels mark 225/279 condition-C outputs as template fallbacks. Post-rewrite verdicts were not saved, and six outputs retained model narration when no nonempty verified template was available. The proxy has F1 = 0.60 against judgments based on extracted context rather than rendered slides. These are observed system-level proxy rates, not component attribution or evidence that hallucination is eliminated.

Generic ham/spam labels do not define a financial-fraud blocking policy. We present a 1,393-message policy-aligned Indian SMS benchmark in Hindi, Hinglish, and English-India, separating source labels, fraud/non-fraud/unclear adjudication, and allow/review/block actions. GPT-4o and Gemini helped adjudicate source-positive messages, so positive metrics are secondary. Our primary endpoint is false-positive rate (FPR) on 1,294 policy-negatives: 1,222 untouched source negatives plus 72 source positives converted to safe. Under a common minimal prompt, six LLM APIs exceed illustrative 1% FPR gates on Bank and UPI templates. GPT-4o-mini records 77.3% and 56.9% FPR; a fine-tuned 270M XLM-R records 0.0% on both, 0.4% globally, and 0.980 recall. Policy definitions and seven examples reduce GPT-4o to 0.0% Bank/UPI and 1.5% global FPR at 0.949 recall; Claude reaches 1.8% at 0.960 recall. Prompting changes which gate fails. GPT-4o P3 and the AND consensus meet the FPR gates at lower policy-label recall; the local model meets both FPR and recall gates on this benchmark. Because positive labels partly use GPT-4o/Gemini, recall and F1 are secondary, not an overall ranking. We release labels, prompts, parsed decisions, analysis code, and checkpoints without a production or unseen-institution claim.

We describe our system for LitTraceQA (GroundLM @ EMNLP 2026): given a research question, retrieve the relevant papers from a pool of 27,487, cite the page and the table or figure where the answer lives, and answer in a requested format. Our main finding is that evidence grounding is limited by retrieval, not by reading. The page selector put the annotator’s page, which we call the gold page, in front of the model that locates evidence only about half the time (52.6% gold-page recall), while that model, given the page, cited the right one in 45 of the 48 locators it emitted (94%). When the page was missing it rarely said so: of 45 such cases it returned nothing 14 times, a wrong page 24 times, and a correct page 7 times, so the pipeline failed quietly almost twice as often as it failed visibly. Since the failure was that the right page was never shown, the fix is to stop choosing: each retrieved paper fits in the model’s context, so we show it whole. Page ranking survives only as a fallback inside papers too long to fit, which no test-split paper was, and gold-page recall reaches 100% on the papers we can parse. Separately, questions that identify their target by position rather than content, such as "the first author of the 24th reference", are served by parsing rather than retrieval: we resolve the bibliography into an addressable list, which also supplies identifiers the evidence metric scores. The final system scores 0.762 paper F₁, 0.441 evidence F₁ and 0.920 multiple-choice accuracy on the held-out test split. Because the pipeline depends on a closed model without seed control, we release a harness that verifies the paper’s central claims against committed artifacts.

Scientific question answering is useful only when readers can recover the papers and evidence behind an answer. Yet many benchmarks either provide the source document or score only the final response, obscuring failures in literature retrieval and evidence localization. We present LitTraceQA, a literature-grounded question answering benchmark in which a system must return a connected trace: canonical paper identifiers, typed evidence locations, and a checkable answer. The collection contains 4,978 unique questions grounded in 4,859 papers and covers tables, figures, text spans, equations or algorithms, and citation contexts. Of these questions, 64.85% require multiple papers; answers use multiple-choice or structured-table formats; and 61.9% are missed by every memory-only challenger used during construction. We describe a generate-and-ground-challenge pipeline and an evaluation protocol that reports paper retrieval, evidence grounding, answer correctness, and strict joint success separately. LitTraceQA therefore tests whether scientific QA systems can produce answers whose provenance is inspectable, rather than merely plausible.

GoldenViewVQA requires models to jointly answer driving-scene questions and identify the camera view containing the supporting visual evidence, making precise evidence localization as important as answer correctness. We present **CoVeR-VQA**, a training-free multi-stage verification and correction framework for grounded multi-view VQA. Starting from GPT-5.6 zero-shot predictions, CoVeR-VQA progressively applies view-specific verification with Gemini-3.6-Flash, prior-guided joint verification with Claude-Opus-5, and cross-split group-level verification that exploits semantically filtered question groups from shared multi-view scenes and validation-derived prior knowledge. On the official GoldenViewVQA test set, the four-stage CoVeR-VQA pipeline achieves 84.75% Joint Accuracy, improving the GPT-5.6 zero-shot baseline by 13.56 percentage points, while reaching 94.92% Answer Accuracy and 86.44% View Accuracy. The final submitted run achieves 88.14% Joint Accuracy after two additional evaluator-informed post-hoc corrections. Our analysis shows that supporting-view localization remains the primary source of residual errors, highlighting the importance of explicit evidence verification for reliable multi-view multimodal reasoning.

We present a non-visual baseline for GoldenViewVQA, a benchmark for multi-view visual question answering with evidence-source identification. The baseline reaches a joint accuracy of 0.7119 on the hidden test split and uses no visual input. Its view prediction is the fixed label CAM_FRONT, which reaches 0.7797 view accuracy on the test split, while Qwen3-4B predicts the answer from the question, answer options, and non-visual metadata with 0.8475 answer accuracy. These results show that the current benchmark permits substantial performance without image access. We analyze the supporting-view prior and performance across question groups, and discuss benchmark changes that could reduce reliance on non-visual shortcuts. We present this run as a strong non-visual baseline, not as evidence that vision is unnecessary for driving question answering. Code: https://github.com/aravind-mandiga/groundlm-goldenviewvqa.

We present ThousandOfWind’s hybrid system for GroundLM 2026 LitTraceQA, combining traditional retrieval-augmented generation (RAG) with code-agent research. Agents design question-specific search scripts, revise them as evidence emerges, and reconcile their findings with RAG candidates. The required-test configuration starts with independent research and accepts subsequent revisions through field-level confidence gates and table-structure checks. Using GPT-5.6 Terra in GitHub Copilot CLI, the system raises the 71-question test composite from 0.3567 to 0.7374 and multiple-choice accuracy from 18/50 to 50/50. Macro table cell accuracy reaches 0.2698, leaving structured assembly as the main bottleneck. Our analysis explains the limits of row-key-preserving repair and motivates source-supported structural editing.

GoldenViewVQA asks a multimodal model to answer a driving-scene question and identify the synchronized camera view that supports its answer. Our first system, QAVEL, used option blind routing, local image inspection, pairwise view selection, and a final verifier. Tests with GPT-5.6 Luna and Terra showed a recurring problem: extra stages often replaced correct direct predictions, and higher reasoning effort did not repair the cascade. We therefore built Joint Answer-preserving Visual Re-grounding (JAVR) around a GPT-5.6 Sol joint prediction. JAVR freezes the answer, aligns the queried actor or condition across all six views, revisits disputed cameras with target-specific crops, and changes only the view when a restricted comparator certifies an exact, direct, and complete evidence source. On the 55-item development set, JAVR produced a descriptive one item increase over its Sol anchor, from 46 to 47 jointly correct predictions, without changing any answer. Our official 59-item test submission reached 79.66% joint accuracy, 81.36% view accuracy, and 94.92% answer accuracy

Finding a relevant paper is not the same as producing a verifiable answer from it. LitTraceQA requires canonical paper identifiers, exact evidence at the page or object level, and typed answers that match the evaluator. We call the separation between source access and scorer-visible correctness the grounding contract gap. TRACE - Target-Aware Retrieval, Attributed Evidence, and Contract-Constrained Extraction - addresses this gap with target-grouped retrieval, independent typed evidence localization, multimodal table extraction, schema-driven table construction, and fail-closed validation. It indexes 27,487 papers through passage, object, alias, citation, and dense representations while retaining the question target behind each signal. For tables, TRACE predicts the observation unit before extracting values and assembles rows with evaluator-compatible key normalization. Our audited selected clean-track artifact scores 0.760613 on the official 71-question test set, including 0.9728 paper F1, 0.6847 evidence F1, 0.9800 multiple-choice accuracy, 0.5423 table-row F1, and 0.3508 macro cell accuracy. On 11 public-development table records, a clean baseline and coordinate-aware visual fill obtain row F1 of 0.291 and 0.411, respectively; this diagnostic comparison includes fallback outputs and is not an official-test claim. Remaining errors chiefly concern locator, observation-unit, row-key, and source-value identity.

Literature-grounded question answering requires a system to retrieve the right papers, locate answer-bearing content in heterogeneous PDFs, and return an answer whose evidence locators can be restored from the retrieved documents. We describe SAGE, an evidence-first system for LitTraceQA that separates these dependencies. A query-only evidence plan converts a question and its public answer contract into typed evidence obligations. The system retrieves candidate papers from official metadata, acquires and parses their PDFs into provenance-preserving text and object records, reranks record-aware text and image projections, and selects a bounded evidence package with explicit slots, modality, and cross-paper coverage priorities. A multimodal generator then answers with the fixed package, while deterministic normalization permits only candidate paper IDs and locators recoverable from selected parser records. Overall, SAGE adopts an evidence-first design that separates paper retrieval, fine-grained evidence acquisition, set-level selection, and grounded answer generation while preserving provenance throughout the pipeline. On the official evaluation, the system performs strongly in paper retrieval and multiple-choice answering, whereas fine-grained evidence grounding and structured table prediction remain more challenging.

We describe team OdeD’s submission to LitTraceQA, the literature-grounded question answering task at GroundLM 2026. The task scores three coupled outputs: paper identifiers, coarse evidence locators, and answers in multiple-choice or table form. Our best official score on the 71-question held-out test split is 0.7649, against 0.4563 for our first submission and 0.5519 for our best fully automated one. Little of that improvement came from the retrieval-and-reading pipeline we began with; most of it came from two other sources. Reading the released evaluator showed the metric to be more asymmetric than it appears: paper F1 and evidence F1 carry weight 1/3 each while each of the three answer metrics carries 1/9. The evidence key is also a coarse tuple, so a correct value on a correct page scores zero when the source type or the visible object id is wrong. The 55-example development split then served to recover the dataset’s annotation conventions rather than as a tuning set, and our own submission history became a measuring instrument. Each macro metric is a mean of a small number of rational per-question values, so a controlled change to one prediction file often admits a single arithmetic explanation. Two predicted deltas returned exact to four decimals. We report the conventions this recovered, six heuristics it refuted, four bugs it exposed in our own verifiers, and the limitation that much of our final score reflects per-question auditing and leaderboard feedback rather than a system that would generalise.

We describe tus-nlp’s Lit3R (Retrieve–Relate–Read) system for LitTraceQA, a shared task for literature-grounded question answering that requires systems to retrieve relevant papers, identify supporting evidence, and generate answers. Lit3R combines off-the-shelf retrieval, reranking, and large language model (LLM) components without task-specific training. The retriever iteratively combines BM25-based sparse and dense retrieval, cross-encoder reranking, and LLM-based verification, and complements retrieval based on the question with paper-to-paper expansion. The reader first identifies supporting evidence within individual papers and then synthesizes evidence across papers to produce the final answer and evidence trace. On the official test set, our system ranked 4th on the leaderboard. Our code is available at https://github.com/tus-ist-nlp/littraceqa.

An autonomous vehicle has the privilege of abundant data drawn from many sources, but abundance raises a question of provenance: when we care about one specific event in the scene, which source actually carries the evi dence for it? Identifying that source matters be yond attribution, because a system that weighs every channel at once lets the irrelevant ones di lute the decision, and more input is not the same as more information. GoldenViewVQA poses this question in a controlled form, asking a sys tem to name which of six synchronised vehicle cameras carries the evidence for a question, and to answer that question. We introduce a novel two-stage system built on symbolic scene serialisation. We sweep all six views with a de tector fine-tuned on nuScenes, back-project ev ery detection onto the ground plane, associate detections across cameras into single objects that carry an explicit view-set, and render the scene as text. A language model then names the supporting view and the answer, reading either the serialisation alone (System 1) or the serialisation together with the six raw images (System 2). The pixels help the system locate evidence, not answer questions: System 2 lifts view macro-accuracy to 0.68, against 0.38 for System 1 and 0.14 for an always-CAM_FRONT baseline. Our best run reaches 0.78 joint accu racy, 0.83 view accuracy and 0.65 view macro accuracy on the official test split. Answer accu racy already reaches 0.95 with no image at all: option phrasing alone largely solves the answer head, leaving nearly all the measurable signal in the view head.

Efficient training and adaptation are central to deploying language models under resource constraints. We present a controlled comparison of six parameter-efficient fine-tuning (PEFT) methods—LoRA, QLoRA, DoRA, PiSSA, LoftQ, and AdaLoRA—across the full cross-product of three base models (Qwen2.5-1.5B, Qwen2.5-7B, Llama-3-8B) and two mathematical reasoning benchmarks (GSM8K, MATH), with every configuration trained on MetaMathQA under one identical budget and reported as mean ± standard deviation over three seeds. Qwen2.5-1.5B + PiSSA reaches 71.40 ± 0.27% on GSM8K, exceeding the unadapted 7B base model (68.50 ± 0.42%; Welch’s t(3.4) = 10.06, p = 0.001). Significance testing shows that the method ordering—PiSSA > DoRA > LoRA ≈ AdaLoRA ≈ LoftQ ≈ QLoRA—is identical across all three base models (Kendall’s τ = 1.0), but that only the PiSSA and DoRA gains over LoRA survive multiple-comparison correction: the LoRA/QLoRA/LoftQ/AdaLoRA cluster is statistically indistinguishable at n=3. Four-bit QLoRA cuts peak training VRAM by up to 69.6% (72.8% with PagedAdamW), within one point of the 68.7% predicted by a static-memory model, while PagedAdamW trades 15–20% throughput for memory safety. We add rank and target-module ablations, a GSM8K error taxonomy, and the finding that PiSSA’s spectral initialization at r=16 outperforms LoRA at r=64, and we release the training, evaluation, and profiling code with the exact configuration for every run.

The first GroundLM shared tasks evaluate whether language models can identify the evidence behind an answer, rather than merely produce a correct answer in isolation. GoldenViewVQA requires a model to answer a multiple-choice question about six synchronized driving-camera views and identify the supporting view. LitTraceQA requires systems to retrieve scientific papers, locate answer-bearing evidence, and return multiple-choice or structured-table answers. We summarize the task design, submitted systems, official results, and recurrent failure modes. Among teams that submitted system reports, the best official joint accuracy on GoldenViewVQA is 0.8814. Answer accuracy is consistently higher than view accuracy, while macro view accuracy remains lower than micro accuracy, exposing both attribution errors and view imbalance. On the required LitTraceQA test set, the best score among teams that submitted system reports is 0.8002. Paper retrieval approaches saturation for several systems, but evidence F1 and structured-table accuracy remain substantially lower. Across both tasks, the central finding is consistent: producing a correct answer is easier than recovering the exact evidence representation required to justify it. We also report an optional 4,901-question LitTraceQA diagnostic split, whose unverified labels make it useful for stress testing but not for ranking systems on the main task.