DeltaMem: Towards Agentic Memory Management via Reinforcement Learning

Qi Zhang, Shen Huang, Liuchu, Shouqing Yang, Xiaomeng Hu, Pengjun Xie, Haobo Wang, Junbo Zhao


Abstract
Recent advances in persona-centric memory demonstrate the promise of multi-agent systems for managing persona memory, especially in conversational scenarios. However, their complex pipelines often incur information loss and poor cross-scenario robustness, leading to suboptimal performance. In this paper, we propose DeltaMem, an agentic memory management system that formulates persona-centric memory management as an end-to-end task within a single-agent setting. To further improve the performance of our agentic memory manager, we draw inspiration from the evolution of human memory and synthesize a user-assistant dialogue dataset along with corresponding operation-level memory updating labels. Building on this, we introduce a novel Memory-based Levenshtein Distance to formalize the memory updating reward, and propose a tailored reinforcement learning framework to further enhance the management capabilities of DeltaMem. Extensive experiments show that both training-free and RL-trained DeltaMem outperform all product-level baselines across diverse long-term memory benchmarks, including LoCoMo, HaluMem, and PersonaMem.
Anthology ID:
2026.emnlp-main.1018
Volume:
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Month:
October
Year:
2026
Address:
Budapest, Hungary
Editors:
Hung-Yi Lee, Sunipa Dev, Jessy Li, Desmond Elliott
Venue:
EMNLP
SIG:
SIGDAT
Publisher:
Association for Computational Linguistics
Note:
Pages:
23390–23417
Language:
URL:
https://aclanthology.org/2026.emnlp-main.1018/
DOI:
Bibkey:
Cite (ACL):
Qi Zhang, Shen Huang, Liuchu, Shouqing Yang, Xiaomeng Hu, Pengjun Xie, Haobo Wang, and Junbo Zhao. 2026. DeltaMem: Towards Agentic Memory Management via Reinforcement Learning. In Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing, pages 23390–23417, Budapest, Hungary. Association for Computational Linguistics.
Cite (Informal):
DeltaMem: Towards Agentic Memory Management via Reinforcement Learning (Zhang et al., EMNLP 2026)
Copy Citation:
PDF:
https://aclanthology.org/2026.emnlp-main.1018.pdf