OrthoEdit: Principled and Stable Knowledge Editing via Orthogonal Subspace Projection

Shanbao Qiao, Xuebing Liu, Akshat Gupta, Seung-Hoon Na


Abstract
Large language models (LLMs) encode extensive factual knowledge through pretraining, yet often require targeted updates to correct errors, incorporate new information, or revise outdated facts. Recent approaches to knowledge editing, such as projection-based constraints, parameter pruning, and regularization, have proven effective in improving editing accuracy and stability. However, these methods often fail to maintain a clear separation between new edits and existing knowledge, leading to interference and degradation over time. We propose OrthoEdit, a principled framework for stable and scalable knowledge editing that ensures each parameter update is orthogonal to both pre-existing and previously edited knowledge, remains strictly non-interfering and preserving the integrity of prior edits. OrthoEdit enables exact subspace control through three coordinated steps: progressive null space refinement, principal subspace extraction, and orthogonal projection. This yields compact and well-aligned updates that systematically satisfy all accumulated constraints. Comprehensive experiments across diverse models and benchmarks demonstrate that OrthoEdit consistently enhances editing accuracy and robustness while preserving general capabilities—even through extended sequences of batched edits. Our code is available at https://github.com/JoveReCode/OrthoEdit.git.
Anthology ID:
2026.tacl-1.51
Volume:
Transactions of the Association for Computational Linguistics, Volume 14
Month:
Year:
2026
Address:
Cambridge, MA
Venue:
TACL
SIG:
Publisher:
MIT Press
Note:
Pages:
1140–1162
Language:
URL:
https://aclanthology.org/2026.tacl-1.51/
DOI:
10.1162/tacl.a.700
Bibkey:
Cite (ACL):
Shanbao Qiao, Xuebing Liu, Akshat Gupta, and Seung-Hoon Na. 2026. OrthoEdit: Principled and Stable Knowledge Editing via Orthogonal Subspace Projection. Transactions of the Association for Computational Linguistics, 14:1140–1162.
Cite (Informal):
OrthoEdit: Principled and Stable Knowledge Editing via Orthogonal Subspace Projection (Qiao et al., TACL 2026)
Copy Citation:
PDF:
https://aclanthology.org/2026.tacl-1.51.pdf