Model editing aims to update large language models (LLMs) with new knowledge without full retraining, but existing methods mainly focus on fact-level knowledge. Rules that support mathematical and scientific reasoning must remain consistent across formulas, descriptions, and instances, making localized updates less effective for rule-level editing. In this paper, we construct RuleEdit-200, a manually verified benchmark of 200 mathematical and physical rules, each instantiated in formula, description, and instance forms. Using this benchmark, we conduct fine-grained causal tracing on four LLMs and identify a form-specific layer-wise separation of rule knowledge: formulas and descriptions are associated with earlier layers, whereas instances rely on a distinct set of middle-layer representations. This finding suggests that rule-level editing requires separate updates for different forms. Based on this insight, we propose Distributed Multi-Layer Editing (DMLE), which updates formulas, descriptions and instances at their corresponding layer ranges. Experiments on RuleEdit-200 show that DMLE remains competitive on standard editing metrics while substantially improving rule-level editing, with average gains of 18.22% and 61.00% in instance portability and rule understanding over the best baseline on each model.

Multimodal Large Language Models (MLLMs) have demonstrated remarkable capabilities by integrating visual and textual inputs, yet modality alignment remains one of the most challenging aspects. Current MLLMs typically rely on simple adapter architectures and pretraining approaches to bridge vision encoders with large language models (LLM), guided by image-level supervision. We identify this paradigm often leads to suboptimal alignment between modalities, significantly constraining the LLM’s ability to properly interpret and reason with visual features particularly for smaller language models. To address this fundamental limitation, we propose Supervised Embedding Alignment (SEA), a token-level supervision alignment method that enables more precise visual-text alignment during pretraining. SEA introduces minimal computational overhead while preserving language capabilities and substantially improving cross-modal understanding. Our comprehensive analyses reveal critical insights into the adapter’s role in multimodal integration, and extensive experiments demonstrate that SEA consistently improves performance across various model sizes, with smaller models benefiting the most (average performance gain of 7.61% for Gemma-2B). This work establishes a foundation for developing more effective alignment strategies for future multimodal systems.