Danyang Zhao
Author directory2026
Unlocking Latent Medical Reasoning in LLMs via Inference-Time Representation and Prefix Interventions
Ming Ma | Jiayun Li | Sendong Zhao | Yanrui Du | Yuzheng Fan | Danyang Zhao | Bing Qin | Ting Liu
Findings of the Association for Computational Linguistics: EMNLP 2026
Ming Ma | Jiayun Li | Sendong Zhao | Yanrui Du | Yuzheng Fan | Danyang Zhao | Bing Qin | Ting Liu
Findings of the Association for Computational Linguistics: EMNLP 2026
Large language models (LLMs) have acquired substantial medical knowledge and reasoning skills during pretraining, yet eliciting these capabilities typically requires scarce rationales or costly post-training. To address this gap, we introduce **RIPT**, a lightweight framework for data-efficient endogenous capability activation. It explicitly decomposes this objective into two complementary components: reasoning enhancement and medical knowledge elicitation. For the former, we extract steering vectors from hidden activations on a small set of high-quality paired reasoning and vanilla responses to shape LLMs’ reasoning behavior. For the latter, we obtain prefix vectors via prefix tuning on simple medical QA pairs to elicit domain-specific knowledge. At inference, RIPT freezes the backbone LLM and combines steering-vector injection with prefix-vector conditioning. Rather than injecting external knowledge, RIPT aims to unlock medical reasoning capabilities already acquired by LLMs during pretraining. Experiments under limited-resource settings show that RIPT consistently improves diverse backbone LLMs and benchmarks, suggesting a generalizable pathway for lightweight medical reasoning enhancement.
Toward Secure Tuning: Mitigating Security Risks from Instruction Fine-Tuning
Yanrui Du | Fenglei Fan | Sendong Zhao | Jiawei Cao | Ming Ma | Danyang Zhao | Shuren Qi | Ting Liu | Bing Qin
Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
Yanrui Du | Fenglei Fan | Sendong Zhao | Jiawei Cao | Ming Ma | Danyang Zhao | Shuren Qi | Ting Liu | Bing Qin
Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
Instruction Fine-Tuning (IFT) has emerged as a critical technique for customizing Large Language Models (LLMs) to meet diverse downstream applications. However, recent studies have revealed that IFT can compromise the built-in security mechanisms of LLMs, thereby posing significant security risks. Although defense methods targeting various training stages have been proposed, they either face challenges in practical deployment or exhibit instability and limited performance gains. In our study, we propose a novel SWAT method that introduces a key idea: shifting more of the learning burden onto security-robust parameters. To this end, our study investigates how module-level parameters affect LLMs’ internal security feature space, aiming to uncover robustness patterns in parameters. Guided by this analysis, we identify a robust module set (Mods_Rob) that exhibits minimal effects on LLMs’ security feature space. Leveraging this insight, SWAT proceeds in two phases: (1) a warm-up phase that preferentially trains Mods_Rob to learn low-level features with minimal security risk, followed by (2) standard tuning to achieve optimal task performance. Across diverse knowledge-intensive datasets, scenarios, and LLMs, SWAT substantially reduces security risks without sacrificing task performance gains.