Diagnosing and Mitigating Evidence Misgrounding in Multimodal Knowledge Editing

Zhiwu Wang, Yichen Wu, Jinqian Yang, Yizhou Yang, Renzhen Wang, Qian Zhao, Deyu Meng


Abstract
Multimodal knowledge editing aims to update visually grounded facts in Multimodal Large Language Models (MLLMs) while preserving unrelated text and visual behaviors. However, standard answer-level metrics can hide evidence misgrounding: an edit may produce the target answer on the original image yet remain sensitive to incidental context rather than the intended visual evidence. To examine this issue in a controlled setting, we construct WATERBIRD-EDIT, an analysis set that decouples bird habitat labels from background context and measures whether edited predictions remain stable when the target evidence is preserved but the background changes. Motivated by this diagnosis, we propose VICE (Visual In-Context Editing), a retrieval-based framework that stores edits in external memory and applies them through visual in-context demonstrations designed to expose task-relevant evidence. Experiments on three standard benchmarks show that VICE achieves strong lifelong editing performance, while results on WATERBIRD-EDIT show that it substantially reduces sensitivity to the controlled background shifts.
Anthology ID:
2026.findings-emnlp.621
Volume:
Findings of the Association for Computational Linguistics: EMNLP 2026
Month:
October
Year:
2026
Address:
Budapest, Hungary
Editors:
Hung-Yi Lee, Sunipa Dev, Jessy Li, Desmond Elliott
Venue:
Findings
SIG:
SIGDAT
Publisher:
Association for Computational Linguistics
Note:
Pages:
13048–13064
Language:
URL:
https://aclanthology.org/2026.findings-emnlp.621/
DOI:
Bibkey:
Cite (ACL):
Zhiwu Wang, Yichen Wu, Jinqian Yang, Yizhou Yang, Renzhen Wang, Qian Zhao, and Deyu Meng. 2026. Diagnosing and Mitigating Evidence Misgrounding in Multimodal Knowledge Editing. In Findings of the Association for Computational Linguistics: EMNLP 2026, pages 13048–13064, Budapest, Hungary. Association for Computational Linguistics.
Cite (Informal):
Diagnosing and Mitigating Evidence Misgrounding in Multimodal Knowledge Editing (Wang et al., Findings 2026)
Copy Citation:
PDF:
https://aclanthology.org/2026.findings-emnlp.621.pdf