Jingmin Xin
Author directory2026
Subspace Alignment Score: Locating Domain-Relevant Important Parameters in Large Language Models
Le Wu | Jiayi Wu | Jixiang Xiong | Mengfa Mo | Jingmin Xin
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Le Wu | Jiayi Wu | Jixiang Xiong | Mengfa Mo | Jingmin Xin
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Although large language models (LLMs) possess powerful general capabilities, their insufficient domain expertise limits their application in high-reliability scenarios. The sparse activation patterns of neurons in LLM feed-forward layers suggest that structured fine-tuning strategies for domain adaptation can be constructed by identifying important parameters, thereby enhancing domain-specific performance while preserving general capabilities. However, due to high computational costs or weak scalability, current studies struggle to support large-scale and fine-grained domain-relevant parameter analysis. To address this, we investigate the distribution patterns and hierarchical evolution patterns of domain knowledge within the hidden state space of LLMs, and propose a novel parameter evaluation method: Subspace Alignment Score. Intervention experiments across three models and six domains illustrate that, compared with baseline methods, our method demonstrates superior performance in locating domain-relevant important parameters. Furthermore, structured fine-tuning strategies based on the important parameters achieve performance comparable to full fine-tuning while substantially reducing training costs, which is helpful for understanding efficient and interpretable domain adaptation. Our code is available at https://github.com/lewu0723/subspace-alignment-score.