Safe and Sufficient Caching: Validated Evidence Reuse for Production Retrieval-Augmented Generation

Akshay Verma, Siddharth Pillai, Sambit Das, Deepak Gupta


Abstract
Production Retrieval-Augmented Generation (RAG) systems serve repeated queries over evolving document corpora under user-specific access constraints. Standard pipelines perform full retrieval and generation for every request, wasting computation on recurring queries. Naive caching reduces latency but introduces safety failures: stale evidence from updated documents, unauthorized content crossing access boundaries, and superseded passages no longer reflecting source truth. These violations are invisible on academic benchmarks yet unacceptable in enterprise deployment. We introduce SSC (Safe and Sufficient Caching), a validated evidence reuse framework for production RAG. SSC caches evidence chunks (not generated answers) with perchunk provenance metadata, and enforces hard validity constraints before reuse: access-control gating ensures user authorization, and freshness verification confirms source-document currency. Only evidence passing both checks enters a utility-based ranking stage balancing relevance, source trust, historical utility, and serving cost. This separation of safety from efficiency is the central contribution. On WebQuestions and HotpotQA, SSC maintains full quality parity with unrestricted retrieval baselines, confirming zero degradation from evidence reuse. On a proprietary InternalQA workload with role based access control and document versioning, it reduces end-to-end latency by 22% and retrieval cost by 34% while achieving zero invalid reuse, compared to 3.4-9.8% violation rates for standard caching and 0.3-8.2% for TTL-based invalidation. The method is training-free, retriever-agnostic, and deploys as a thin validation layer in existing RAG pipelines.
Anthology ID:
2026.emnlp-industry.75
Volume:
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing: Industry Track
Month:
October
Year:
2026
Address:
Budapest, Hungary
Editors:
Pepa Atanasova, Gholamreza Haffari, Minghan Wang
Venue:
EMNLP
SIG:
SIGDAT
Publisher:
Association for Computational Linguistics
Note:
Pages:
1272–1279
Language:
URL:
https://aclanthology.org/2026.emnlp-industry.75/
DOI:
Bibkey:
Cite (ACL):
Akshay Verma, Siddharth Pillai, Sambit Das, and Deepak Gupta. 2026. Safe and Sufficient Caching: Validated Evidence Reuse for Production Retrieval-Augmented Generation. In Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing: Industry Track, pages 1272–1279, Budapest, Hungary. Association for Computational Linguistics.
Cite (Informal):
Safe and Sufficient Caching: Validated Evidence Reuse for Production Retrieval-Augmented Generation (Verma et al., EMNLP 2026)
Copy Citation:
PDF:
https://aclanthology.org/2026.emnlp-industry.75.pdf