Samarth Vinayaka
Author directory2026
When Should LLM Agents Forget? Memory Filtering Under Query-Critical Resolver Errors
Samarth Vinayaka
Proceedings of the Second Workshop for REsearch on Agent Language Models (REALM 2026)
Samarth Vinayaka
Proceedings of the Second Workshop for REsearch on Agent Language Models (REALM 2026)
LLM memory systems often filter facts classified as stale from the current prompt. We study this query-time context filtering, without deleting stored memories, in memory-conditioned question answering. Across three models on a synthetic benchmark, controlled label corruption moves hard-drop from gains of +0.186 to +0.230 at perfect resolution to losses of −0.176 to −0.181 at the lowest tested reliability. Paired targeted interventions show greater degradation from false-drop than false-keep; retaining evidence can still fail when incorrect labels steer generation. On one frozen 102/102 calibration/test query split, empirical policy selection using labeled calibration outcomes reaches 0.655 accuracy versus 0.641 for always hard-drop and 0.662 for test-informed oracle selection. A question-disjoint MemoryAgentBench replication is inconclusive: gain +0.03, 95% CI [−0.01, +0.08]. These results establish a controlled sensitivity mechanism, not generalization to long-horizon agents. Global-accuracy crossovers require gold labels and are model- and task-specific diagnostics. Context filtering should be cost-sensitive and validated on matched labeled data; label-free retention is an accuracy fallback under the tested errors, subject to separate privacy and retention requirements.
Groundedness Is Not Truth: Evidence-Chain Forensics for Retrieval-Augmented Generation
Samarth Vinayaka
Proceedings of the 1st Workshop on Grounding Language Models: Learning Faithfully and Efficiently (GroundLM 2026)
Samarth Vinayaka
Proceedings of the 1st Workshop on Grounding Language Models: Learning Faithfully and Efficiently (GroundLM 2026)
A single faithfulness score can hide failures that require different repairs: a RAG answer may be grounded in text whose source is stale, incomplete, contradicted, low-authority, or insufficient. We introduce ContextTrace, a deterministic, local-first SDK and CLI for evidence-chain forensics that separates claim support, citation state, source condition, abstention need, root cause, repair, and regression prevention. We also introduce Groundedness-Truth-Gap, a suite for controlled conformance, while frozen RAGTruth and stress tests serve as failure-safety controls. ContextTrace’s central contribution is an inspectable evidence-chain contract. On a fresh 84-case public-document evaluation, it catches 76 of 77 failure-bearing cases and preserves all seven clean cases (balanced accuracy 0.994; MCC 0.929), the same binary confusion counts as a frozen GPT-4.1-mini judge, while producing typed diagnostic records without additional model calls. The clean-case sample is small, and fine-grained label, root, and unverifiable attribution remains limited. Post-review counterfactuals further show strong dependence on source metadata and temporal cues. We release a blinded 50-case annotation packet and scorer; independent adjudication remains pending. Our claim is reproducible fail-closed diagnosis, not truth verification or broad evaluator superiority.