Knowledge editing is a crucial technique for daily updates in LLMs, requiring a balance between accurately modifying incorrect knowledge and preserving existing information. The recently proposed AlphaEdit method achieves competitive editing performance by updating parameters under null-space constraints. However, our theoretical analysis reveals that AlphaEdit struggles with high knowledge conflicts and inconsistencies during editing. To address this, we propose a new editing method AlphaEdit+, featuring three key improvements: 1) relaxing null-space constraints by adding a matrix perturbation through optimization to resolve conflicts between new and preserved knowledge; 2) introducing a weighting scheme on previously updated knowledge constraints to mitigate conflicts between new and historical editing; 3) developing a value smoothing algorithm to resolve high knowledge inconsistencies. These enhancements collectively ensure robust editing while maintaining model coherence. Comprehensive experiments show that our approach AlphaEdit+ not only resolves the brittleness of the original method on carefully constructed challenging datasets but also outperforms AlphaEdit on existing benchmark datasets.

Knowledge editing updates facts in large language models while preserving unrelated knowledge. Most prior work studies object editing, which modifies the object under a fixed subject–relation pair. We instead study relation replacement editing, where an obsolete relation must be suppressed as a new one is introduced. We systematically study this setting with a dedicated benchmark and method. We introduce ReEditBench to evaluate replacement rather than association addition. Existing editors exhibit two major failures: obsolete-relation retention remains as high as 98.20% after successful edits, and difficult samples often destabilize editing. To address this, we propose R²Edit, a two-stage framework for relation replacement. Its target-smoothed Forget stage reduces obsolete-relation activation without fixed/random pseudo-target bias, while its curriculum-guided Edit stage progressively injects new relations from easy to hard samples. Across multiple LLMs, R²Edit improves replacement success, reduces retention, and preserves generalization. Stress tests on difficult subsets further demonstrate the robustness of the Edit stage. Code is available at: https://github.com/caoxu0109/R2Edit.

Token selection is pivotal for effective LLM post-training. However, existing methods mostly rely on local token-level heuristics, while response-token selection in post-training lacks an optimization-guided valuation proxy. We introduce AlphaToken, a response token valuation framework that decouples valuation into adaptation (promoting target-task learning) and stability (preserving pre-trained capabilities), and makes each objective path-aware by combining the direct-path signal from local token gradients with the downstream causal-path signal in autoregressive generation. Since retention data are typically unavailable, AlphaToken approximates stability via a Fisher-drift proxy anchored at the pre-trained reference model. For efficient computation, we extend Ghost Dot-Product to token-level valuation. AlphaToken masks low-value response tokens during fine-tuning and preference optimization, concentrating training signals on more valuable positions. Experiments show that AlphaToken improves post-training performance and mitigates catastrophic forgetting. Code at https://github.com/luciabcd/Alphatoken.