Danyang Zhao

Author directory

2026

Large language models (LLMs) have acquired substantial medical knowledge and reasoning skills during pretraining, yet eliciting these capabilities typically requires scarce rationales or costly post-training. To address this gap, we introduce **RIPT**, a lightweight framework for data-efficient endogenous capability activation. It explicitly decomposes this objective into two complementary components: reasoning enhancement and medical knowledge elicitation. For the former, we extract steering vectors from hidden activations on a small set of high-quality paired reasoning and vanilla responses to shape LLMs’ reasoning behavior. For the latter, we obtain prefix vectors via prefix tuning on simple medical QA pairs to elicit domain-specific knowledge. At inference, RIPT freezes the backbone LLM and combines steering-vector injection with prefix-vector conditioning. Rather than injecting external knowledge, RIPT aims to unlock medical reasoning capabilities already acquired by LLMs during pretraining. Experiments under limited-resource settings show that RIPT consistently improves diverse backbone LLMs and benchmarks, suggesting a generalizable pathway for lightweight medical reasoning enhancement.
Instruction Fine-Tuning (IFT) has emerged as a critical technique for customizing Large Language Models (LLMs) to meet diverse downstream applications. However, recent studies have revealed that IFT can compromise the built-in security mechanisms of LLMs, thereby posing significant security risks. Although defense methods targeting various training stages have been proposed, they either face challenges in practical deployment or exhibit instability and limited performance gains. In our study, we propose a novel SWAT method that introduces a key idea: shifting more of the learning burden onto security-robust parameters. To this end, our study investigates how module-level parameters affect LLMs’ internal security feature space, aiming to uncover robustness patterns in parameters. Guided by this analysis, we identify a robust module set (Mods_Rob) that exhibits minimal effects on LLMs’ security feature space. Leveraging this insight, SWAT proceeds in two phases: (1) a warm-up phase that preferentially trains Mods_Rob to learn low-level features with minimal security risk, followed by (2) standard tuning to achieve optimal task performance. Across diverse knowledge-intensive datasets, scenarios, and LLMs, SWAT substantially reduces security risks without sacrificing task performance gains.