2026

Lifelong model editing seeks to continuously incorporate new knowledge into Large Language Models (LLMs) without costly retraining. Memory-augmented editors preserve pretrained weights to protect unrelated capabilities, but struggle to balance Generalization to rephrased edit queries and Locality on non-edited knowledge. This challenge becomes particularly severe under dense editing settings, where semantically related edits induce substantial memory interference. To tackle this problem, we propose MH-HoReN, a multi-head Hopfield retrieval framework that performs retrieval across complementary subspaces and combines head-wise matching signals for more reliable memory selection. We further extend Hopfield-based editing into a clustered batched scheme that combines shared codebook memories for related edits with token-level pooling for efficient knowledge storage and query routing. Extensive experiments across diverse LLMs and editing benchmarks demonstrate that MH-HoReN achieves better locality-generalization trade-offs as editing scales increase, improving the sum of Locality and Generalization by more than 12% over existing methods while preserving almost perfect Reliability in sequential editing, alongside 7.7% gains in Overall Performance under batched editing. Our code is available here.

2025

Despite their remarkable progress in multimodal understanding tasks, large vision language models (LVLMs) often suffer from “hallucination”, generating texts misaligned with the visual context. Existing methods aimed at reducing hallucinations through inference time intervention incur a significant increase in latency. To mitigate this, we present SPIN, a task-agnostic attention-guided head suppression strategy that can be seamlessly integrated during inference without incurring any significant compute or latency overhead. We investigate whether hallucination in LVLMs can be linked to specific model components. Our analysis suggests that hallucinations can be attributed to a dynamic subset of attention heads in each layer. Leveraging this insight, for each text query token, we selectively suppress attention heads that exhibit low attention to image tokens, keeping the top-k attention heads intact. Extensive evaluations on visual question answering and image description tasks demonstrate the efficacy of SPIN in reducing hallucination scores up to 2.7x while maintaining F1, and improving throughput by 1.8x compared to existing alternatives.