Latent Personal Memory: Represent personal memory as dynamic soft prompts

Debrup Das, Avinash Amballa, Yashas Malur Saidutta, Vijay Srinivasan, Vivek Kulkarni, Srinivas Chappidi


Abstract
Personalizing large language models (LLMs) requires encoding long-term, user-specific behavioral patterns in a way that is computationally efficient, scalable, and compatible with a frozen base model. We present Latent Personal Memory (LPM), a framework that represents user-specific history as a compact, persistent matrix of latent slots, that are interpretable. A shared cross-attention projection network maps these slots into dynamic, input-conditioned soft prompts that are prepended to the input of a frozen LLM. We evaluate LPM on PersonaMem v1 and LoCOMO benchmarks across Qwen3-1.7B, 4B, and 8B backbones. Results demonstrate that LPM outperforms LoRA and Prompt Tuning by up to 8.8 % and 54.4% in overall accuracy respectively on PersonaMem v1, while reducing KV-cache usage by over 64x. On LoCoMo, LPM matches LoRA’s accuracy while requiring approximately 120x fewer trainable parameters. We also show that LPM’s efficiency grows with context length and outperforms full-context at 128K context length.
Anthology ID:
2026.findings-emnlp.624
Volume:
Findings of the Association for Computational Linguistics: EMNLP 2026
Month:
October
Year:
2026
Address:
Budapest, Hungary
Editors:
Hung-Yi Lee, Sunipa Dev, Jessy Li, Desmond Elliott
Venue:
Findings
SIG:
SIGDAT
Publisher:
Association for Computational Linguistics
Note:
Pages:
13113–13128
Language:
URL:
https://aclanthology.org/2026.findings-emnlp.624/
DOI:
Bibkey:
Cite (ACL):
Debrup Das, Avinash Amballa, Yashas Malur Saidutta, Vijay Srinivasan, Vivek Kulkarni, and Srinivas Chappidi. 2026. Latent Personal Memory: Represent personal memory as dynamic soft prompts. In Findings of the Association for Computational Linguistics: EMNLP 2026, pages 13113–13128, Budapest, Hungary. Association for Computational Linguistics.
Cite (Informal):
Latent Personal Memory: Represent personal memory as dynamic soft prompts (Das et al., Findings 2026)
Copy Citation:
PDF:
https://aclanthology.org/2026.findings-emnlp.624.pdf