Xijian Deng
Author directory2026
Mitigating Memory Interference in Lifelong Model Editing through Multi-Head Hopfield Retrieval
Yue Che | Chen Guoyi | Xijian Deng | Yujie Wu | Jiang Cai | Xuming Ran | Mingkun Xu
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Yue Che | Chen Guoyi | Xijian Deng | Yujie Wu | Jiang Cai | Xuming Ran | Mingkun Xu
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Lifelong model editing seeks to continuously incorporate new knowledge into Large Language Models (LLMs) without costly retraining. Memory-augmented editors preserve pretrained weights to protect unrelated capabilities, but struggle to balance Generalization to rephrased edit queries and Locality on non-edited knowledge. This challenge becomes particularly severe under dense editing settings, where semantically related edits induce substantial memory interference. To tackle this problem, we propose MH-HoReN, a multi-head Hopfield retrieval framework that performs retrieval across complementary subspaces and combines head-wise matching signals for more reliable memory selection. We further extend Hopfield-based editing into a clustered batched scheme that combines shared codebook memories for related edits with token-level pooling for efficient knowledge storage and query routing. Extensive experiments across diverse LLMs and editing benchmarks demonstrate that MH-HoReN achieves better locality-generalization trade-offs as editing scales increase, improving the sum of Locality and Generalization by more than 12% over existing methods while preserving almost perfect Reliability in sequential editing, alongside 7.7% gains in Overall Performance under batched editing. Our code is available here.