Rent-a-RAG: Embedding-Space Watermarks for Auditing Third-Party RAG

Alexandr Goultiaev Tolstokorov, Kyriakos Mouratidis, Javad Dogani, Nikolaos Laoutaris


Abstract
Third-party retrieval-augmented generation (RAG) marketplaces create a new auditing problem: data providers may license corpora to a RAG operator, yet later have no visibility into whether their documents are being reused without compensation. Auditing this misuse is difficult because the operator is non-cooperative, answers are paraphrased by the generator, and one response may combine evidence from many providers. We propose DirBucket, a provider-side semantic watermarking and black-box auditing framework for document-level reuse in multi-provider RAG. DirBucket watermarks documents by meaning-preserving paraphrases whose embeddings are biased toward provider-bucket secret directions, enabling detection from black-box answers while preserving retrieval utility. On a challenging benchmark that reflects mixed-provider reuse under black-box access, DirBucket is the only method that consistently achieves strong target detection with no non-target activation, detecting non-compliance in every audit within 23 audited answers on our primary benchmark. The watermark survives adversarial post-answer laundering, and none of the evaluated evasion strategies simultaneously defeats detection while preserving user-perceived answer quality. Detection transfers unchanged to a second benchmark built from real clinical, cyber-threat-intelligence, and legal provider corpora. These results suggest that embedding-space watermarking can make document reuse in third-party RAG statistically auditable.
Anthology ID:
2026.emnlp-main.1809
Volume:
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Month:
October
Year:
2026
Address:
Budapest, Hungary
Editors:
Hung-Yi Lee, Sunipa Dev, Jessy Li, Desmond Elliott
Venue:
EMNLP
SIG:
SIGDAT
Publisher:
Association for Computational Linguistics
Note:
Pages:
41216–41243
Language:
URL:
https://aclanthology.org/2026.emnlp-main.1809/
DOI:
Bibkey:
Cite (ACL):
Alexandr Goultiaev Tolstokorov, Kyriakos Mouratidis, Javad Dogani, and Nikolaos Laoutaris. 2026. Rent-a-RAG: Embedding-Space Watermarks for Auditing Third-Party RAG. In Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing, pages 41216–41243, Budapest, Hungary. Association for Computational Linguistics.
Cite (Informal):
Rent-a-RAG: Embedding-Space Watermarks for Auditing Third-Party RAG (Tolstokorov et al., EMNLP 2026)
Copy Citation:
PDF:
https://aclanthology.org/2026.emnlp-main.1809.pdf