ShardMemo: Scope-Before-Routing for Agentic Memory Retrieval

Yang Zhao, Chengxiao Dai, Mengying Kou, Yue Xiu, Dusit Niyato


Abstract
Agentic systems accumulate persistent memory across sessions, tools, and tasks, and a later request must retrieve from it under two distinct constraints: which memories it is permitted to access, and which are relevant under a limited search budget. Existing memory systems handle access scope in two flawed ways: applying scope after retrieval wastes probe budget on inadmissible memories, while treating scope as a learned ranking feature makes a hard constraint depend on router quality. We present ShardMemo, an agentic memory system built on scope-before-routing: metadata predicates first identify the admissible shards, and a learned router then selects a small number of them for shard-local approximate nearest neighbor retrieval. This separates hard admissibility from learned relevance ranking, so inadmissible shards cannot consume shard-probe budget. We evaluate on LoCoMo, HotpotQA, and ToolBench, covering conversational, long-context, and procedural memory. Under matched supervision and fixed budgets, ShardMemo improves over a learned router baseline by roughly +3 F1 on LoCoMo; in end-to-end LoCoMo evaluation it improves over the strongest evaluated memory baseline by up to +6.8 F1, with gains on HotpotQA and ToolBench.
Anthology ID:
2026.findings-emnlp.1806
Volume:
Findings of the Association for Computational Linguistics: EMNLP 2026
Month:
October
Year:
2026
Address:
Budapest, Hungary
Editors:
Hung-Yi Lee, Sunipa Dev, Jessy Li, Desmond Elliott
Venue:
Findings
SIG:
SIGDAT
Publisher:
Association for Computational Linguistics
Note:
Pages:
38596–38610
Language:
URL:
https://aclanthology.org/2026.findings-emnlp.1806/
DOI:
Bibkey:
Cite (ACL):
Yang Zhao, Chengxiao Dai, Mengying Kou, Yue Xiu, and Dusit Niyato. 2026. ShardMemo: Scope-Before-Routing for Agentic Memory Retrieval. In Findings of the Association for Computational Linguistics: EMNLP 2026, pages 38596–38610, Budapest, Hungary. Association for Computational Linguistics.
Cite (Informal):
ShardMemo: Scope-Before-Routing for Agentic Memory Retrieval (Zhao et al., Findings 2026)
Copy Citation:
PDF:
https://aclanthology.org/2026.findings-emnlp.1806.pdf