Samarth Vinayaka

Author directory

2026

LLM memory systems often filter facts classified as stale from the current prompt. We study this query-time context filtering, without deleting stored memories, in memory-conditioned question answering. Across three models on a synthetic benchmark, controlled label corruption moves hard-drop from gains of +0.186 to +0.230 at perfect resolution to losses of −0.176 to −0.181 at the lowest tested reliability. Paired targeted interventions show greater degradation from false-drop than false-keep; retaining evidence can still fail when incorrect labels steer generation. On one frozen 102/102 calibration/test query split, empirical policy selection using labeled calibration outcomes reaches 0.655 accuracy versus 0.641 for always hard-drop and 0.662 for test-informed oracle selection. A question-disjoint MemoryAgentBench replication is inconclusive: gain +0.03, 95% CI [−0.01, +0.08]. These results establish a controlled sensitivity mechanism, not generalization to long-horizon agents. Global-accuracy crossovers require gold labels and are model- and task-specific diagnostics. Context filtering should be cost-sensitive and validated on matched labeled data; label-free retention is an accuracy fallback under the tested errors, subject to separate privacy and retention requirements.
A single faithfulness score can hide failures that require different repairs: a RAG answer may be grounded in text whose source is stale, incomplete, contradicted, low-authority, or insufficient. We introduce ContextTrace, a deterministic, local-first SDK and CLI for evidence-chain forensics that separates claim support, citation state, source condition, abstention need, root cause, repair, and regression prevention. We also introduce Groundedness-Truth-Gap, a suite for controlled conformance, while frozen RAGTruth and stress tests serve as failure-safety controls. ContextTrace’s central contribution is an inspectable evidence-chain contract. On a fresh 84-case public-document evaluation, it catches 76 of 77 failure-bearing cases and preserves all seven clean cases (balanced accuracy 0.994; MCC 0.929), the same binary confusion counts as a frozen GPT-4.1-mini judge, while producing typed diagnostic records without additional model calls. The clean-case sample is small, and fine-grained label, root, and unverifiable attribution remains limited. Post-review counterfactuals further show strong dependence on source metadata and temporal cues. We release a blinded 50-case annotation packet and scorer; independent adjudication remains pending. Our claim is reproducible fail-closed diagnosis, not truth verification or broad evaluator superiority.