Jingmin Xin

Author directory

2026

Although large language models (LLMs) possess powerful general capabilities, their insufficient domain expertise limits their application in high-reliability scenarios. The sparse activation patterns of neurons in LLM feed-forward layers suggest that structured fine-tuning strategies for domain adaptation can be constructed by identifying important parameters, thereby enhancing domain-specific performance while preserving general capabilities. However, due to high computational costs or weak scalability, current studies struggle to support large-scale and fine-grained domain-relevant parameter analysis. To address this, we investigate the distribution patterns and hierarchical evolution patterns of domain knowledge within the hidden state space of LLMs, and propose a novel parameter evaluation method: Subspace Alignment Score. Intervention experiments across three models and six domains illustrate that, compared with baseline methods, our method demonstrates superior performance in locating domain-relevant important parameters. Furthermore, structured fine-tuning strategies based on the important parameters achieve performance comparable to full fine-tuning while substantially reducing training costs, which is helpful for understanding efficient and interpretable domain adaptation. Our code is available at https://github.com/lewu0723/subspace-alignment-score.