Existing continual knowledge editing methods suffer from cognitive dissonance: after an edit, the model answers the edit prompt correctly but reverts to the old fact under complex equivalent queries. This means old and new facts coexist rather than the old being replaced. The root cause of cognitive dissonance is that existing methods fit the new fact on the target prompt without overwriting the old one. In constrained methods, static null-space projectors lock outdated representations into the protected subspace, forcing updates into orthogonal directions. To resolve cognitive dissonance, we propose P-Space (Puncturing the Null-Space to Resolve Cognitive Dissonance in Continual Knowledge Editing), which at each edit step (1) punctures the protected covariance to unprotect the outdated representation, (2) overwrites the old mapping in closed form along its original direction under the punctured projector, and (3) anchors the new fact into the covariance with adaptive scaling so that future edits preserve it. Experiments on two datasets show that P-Space outperforms existing methods while preserving general capabilities.

Mixture of Experts (MoE) has great potential in scaling up the capacity of large models while maintaining low computational costs. However, existing methods mainly reduce redundancy at the expert level while overlooking fine-grained neuron-level redundancy and expert collaboration. In this paper, we propose an Information Bottleneck based MoE (IBMoE) for parameter-efficient fine-tuning, which reduces neuron-level redundancy within each expert and fosters internal collaboration among all experts. Specifically, IBMoE equips each expert with learnable queries to capture task-specific characteristics and employs sparse neuronal activation to dynamically select the most relevant neurons, reducing redundant activations across tasks. In addition, a diversity constraint is imposed among experts to maximize their knowledge diversity, enabling them to cooperate more effectively. Extensive experiments across multiple benchmarks show that IBMoE consistently outperforms strong baselines while activating significantly fewer parameters. In addition, IBMoE reduces inference latency by 61.7% and the memory footprint of adapters by 48.2% compared with previous MoE-based adaptation methods, while achieving comparable training costs.

Knowledge editing is a promising method for updating Large Language Models efficiently. However, previous studies often suffer from poor specificity in continual editing, as they typically focus on single edits or preventing knowledge forgetting. To address this, we propose TamEdit, a trajectory-aware meta-learning method that preserves specificity for continual knowledge editing. TamEdit unifies three levels: Inner Optimization performs multi-step fast fine-tuning on the single edit; Trajectory-based Editing unifies continual edits with a growing memory; and Outer Optimization leverages meta-learning to distill cross-task strategies for preserving specificity. By capturing the relationships between different single edits within the trajectory, our method learns how to effectively avoid specificity drift. Experiments across multiple LLMs show TamEdit significantly outperforms baselines in continual editing, improving specificity by 14.81% with fast speed (requiring only 8.84% of the time cost of most baselines), while preserving general capabilities.

Topic modeling has been widely used for discovering the latent semantic structure of documents, but most existing methods learn topics with a flat structure. Although probabilistic models can generate topic hierarchies by introducing nonparametric priors like Chinese restaurant process, such methods have data scalability issues. In this study, we develop a tree-structured topic model by leveraging nonparametric neural variational inference. Particularly, the latent components of the stick-breaking process are first learned for each document, then the affiliations of latent components are modeled by the dependency matrices between network layers. Utilizing this network structure, we can efficiently extract a tree-structured topic hierarchy with reasonable structure, low redundancy, and adaptable widths. Experiments on real-world datasets validate the effectiveness of our method.