While large language models (LLMs) show immense potential in specialized domains, their adaptation is often hindered by limited access to domain-specific data and the high computational cost of model training. To address these challenges, we propose a lightweight knowledge injection framework. Instead of sharing raw data directly, our approach fine-tunes a small model and transfers its knowledge vectors to the target LLM for knowledge injection, thereby reducing data exposure and computational overhead. To mitigate the knowledge loss incurred during the injection process, we further introduce an iterative knowledge calibration strategy. We theoretically bound the error of the transformation matrix and prove the stability and linear convergence of the calibration process. Comprehensive experiments demonstrate that our method significantly outperforms parameter-efficient baselines and achieves performance comparable to full-parameter fine-tuning, while consuming only 28% of the GPU memory and 3% of the raw training data. Code is available at https://github.com/Yuhanleeee/LiSKI.

Multi-style outline controllable generation is crucial for multiple applications, including document semantic structuring and retrieval-augmented generation.The great success of preference alignment approaches encourages their application in controllable generation tasks.However, these attempts encounter several limitations: (1) response pair requirements, (2) substantial computation costs, and (3) insufficient exploitation of fine-grained preference signals.To address these problems, we propose a token-level preference self-alignment optimization, named TKPO, for outline controllable generation. TKPO extends the Bradley-Terry model from pair-wise to list-wise comparison, which is further applied at the token level for fine-grained preference signal utilization. In comparison to the representative methods, e.g., DPO, TKPO does not require response pairs; instead, we propose a controllable attributes-driven method to construct reject samples for self-alignment. Additionally, TKPO optimizes only the base model, thereby avoiding additional memory usage and substantial computational costs.We curate two outline controllable generation datasets with regard to language style and level-of-detail.Extensive experiments demonstrate that TKPO outperforms DPO by up to 19.28% in performance while requiring only 56.25% in training time.We release the code and datasets resources at https://github.com/WHUIR/TKPO.