Jizhi Zhang
Author directoryOther people with similar names: Jizhi Zhang
Unverified author pages with similar names: Jizhi Zhang
2026
RULE-EDIT: Benchmarking Rule-Level Knowledge Editing in Large Language Models
Jizhi Zhang | Dylan Xinming Hou | Zhaoyi Li | Qishun Yang | Mahardika Krisna Ihsani | Muhammad Asif Ali | Gus Xia | Lijie Hu
Findings of the Association for Computational Linguistics: EMNLP 2026
Jizhi Zhang | Dylan Xinming Hou | Zhaoyi Li | Qishun Yang | Mahardika Krisna Ihsani | Muhammad Asif Ali | Gus Xia | Lijie Hu
Findings of the Association for Computational Linguistics: EMNLP 2026
Knowledge editing seeks to update language models without full retraining. While most prior work focuses on isolated factual or instance-level edits, we study a more structured setting: mathematical rules as a controlled stress test for abstract symbolic rule editing. We introduce RULE-EDIT, the first benchmark explicitly designed for editing and evaluating rule-level knowledge in LLMs. Our benchmark investigates two key questions: (1) whether edits to abstract rules propagate to derived instances, and (2) whether edits remain consistent across different representations of the same rule. To evaluate these aspects, we propose two new metrics: Instance Portability and Rule Understanding. Experiments on strong open-source LLMs with representative editing methods show that, although models can often overwrite formula-level knowledge, they struggle to transfer edits to rule-derived instances and to maintain consistency across symbolic and natural-language forms. For example, several methods achieve nearly 100% reliability on direct rule queries, while Instance Portability never exceeds 52% and Rule Understanding remains below 26%. Overall, RULE-EDIT provides a controlled benchmark for studying whether current knowledge editing approaches truly modify abstract symbolic rules in LLMs, rather than merely altering surface outputs.