Remember, Verify, or Ask? Evaluating Memory Decisions at Reuse Time

Baichuan Li, Junyi Yao, Zihao Zheng


Abstract
Persistent memory can personalize an LLM agent, but an incorrect durable update can distort future behavior. We introduce the Memory–Clarification Boundary benchmark (MCB), which supplies an interaction-derived candidate and a later reuse context and asks whether to persist it, use it locally, verify it, or clarify it with the user. The release contains 140 primary scenarios, 70 evidence-flip contrasts, and a 36-item real-dialogue-derived extension. On 70 adjudicated primary-test items, verification recall exceeds clarification recall in all 12 label conditions across four checkpoints. However, only three comparisons remain significant after an exploratory Fisher test with Holm correction, and none involve Claude. A category oracle attains .800 accuracy, exposing strong scenario-category structure. Original label and tool-selection conditions agree on only 23–57% of items, although their differing prompts prevent attributing this solely to the interface. The naturalistic extension is descriptive: it contains only two clarification items and evaluates local models only. MCB measures reuse-time decision selection under a conservative policy; storage backends, matched policy/tool interventions, and downstream utility remain unevaluated.
Anthology ID:
2026.nora-1.2
Volume:
Proceedings of the 3rd Workshop on KNOwledge GRaphs Agentic Systems Interplay (NORA 2026)
Month:
November
Year:
2026
Address:
Hengqin, China
Editors:
Btissam Er-Rahmadi, Sébastien Montella, Damien Graux, André Melo, Hajira Jabeen
Venues:
NORA | WS
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
15–25
Language:
URL:
https://aclanthology.org/2026.nora-1.2/
DOI:
Bibkey:
Cite (ACL):
Baichuan Li, Junyi Yao, and Zihao Zheng. 2026. Remember, Verify, or Ask? Evaluating Memory Decisions at Reuse Time. In Proceedings of the 3rd Workshop on KNOwledge GRaphs Agentic Systems Interplay (NORA 2026), pages 15–25, Hengqin, China. Association for Computational Linguistics.
Cite (Informal):
Remember, Verify, or Ask? Evaluating Memory Decisions at Reuse Time (Li et al., NORA 2026)
Copy Citation:
PDF:
https://aclanthology.org/2026.nora-1.2.pdf
Supplementary material:
 2026.nora-1.2.supplementary_material.zip
Source files:
 2026.nora-1.2.source_files.zip