Large Reasoning Models(LRMs) have achieved significant breakthroughs over prior large language models (LLMs), but they also entail greater potential safety risks. Existing alignment methods often remain at a shallow level of protection, making them insufficient to address deeper risks and strategic attacks in complex reasoning processes. To bridge this gap, we move beyond the conventional paradigm that treats safety alignment merely as a preventive measure to reduce harmful outputs. Drawing inspiration from human-like introspection and self-correction, we propose Self-Reflection, a technique that introduces a special Self-Reflection token, enabling LRMs to perform Self-Reflection during generation and recover from harmful outputs. Our approach integrates seamlessly into standard post-training paradigms , further enhancing both helpfulness and safety. The experimental results demonstrate that models trained with Self-Reflection not only consistently outperform the baseline in terms of safety (reducing the HCR from 13.8% to 4.1%, nearly a threefold improvement over mainstream approaches), but also achieve substantial advantages in both helpfulness and the safety–helpfulness balance. More importantly, under evaluations involving various adversarial attacks, including a specially designed adaptive attack, the Self-Reflection mechanism significantly enhances model safety without targeted adversarial training.Notice: This paper contains harmful content.

Large language models (LLMs) often hallucinate in question answering (QA) tasks due to a lack of factual knowledge. While integrating knowledge graphs (KGs) with LLMs has alleviated this issue, existing methods suffer from poor generalization or low reasoning efficiency, and critically, they overlook the learning and reuse of reasoning paths from past experiences. To address these challenges, we introduce Thought-Action Graph (TAG), a structured repository of reasoning experiences. TAG decomposes successful LLM-KG interaction trajectories into fine-grained semantic operators, which are stored in TAG constructed by the thought layer and action layer. Building upon TAG, we propose a novel KGQA paradigm — TAG-Reasoning (TAGR). TAGR first retrieves and assembles reasoning blueprints from TAG, and then guides LLM to efficiently execute on KG according to them. Through this approach, TAGR transforms the computationally expensive online exploration process of LLMs into an offline process of TAG retrieval and assembly. Experimental results on multiple KGQA benchmarks demonstrate that TAGR significantly outperforms state-of-the-art methods across key metrics, while drastically reducing the number of LLM calls and generated tokens. This work opens new avenues for building continual learning, efficient, and faithful KGQA systems.

Safety alignment plays a vital role in ensuring the reliability of Large Language Models (LLMs) by mitigating risks from malicious queries. To avoid the high computational costs of post-training, recent research has increasingly shifted toward inference-time alignment via continuous activation steering.

However, the geometric entanglement of harmful and benign technical representations within these continuous spaces impedes precise calibration, inevitably trapping models in a severe dilemma between jailbreak vulnerability and benign over-refusal. To address this, we propose Codebook-Orthogonal Direction Alignment (CODA), an inference-time framework that projects hidden states onto a vector-quantized codebook to serve as a discrete control interface.

Specifically, to break continuous directional entanglement, CODA estimates steering directions over discrete codebook anchors. To eliminate the geometric interference triggering over-refusals, CODA softly orthogonalizes the safety direction with respect to benign technical intent.

Finally, to prevent rigid over-correction, CODA forwards the decoupled vectors to yield dynamically adjusted steering strengths during generation via risk gating and exponential time decay. Experiments across a diverse set of open-weight LLMs demonstrate that CODA consistently yields a stronger safety–utility trade-off than continuous steering baselines, indicating that discrete representations provide a more effective mechanism for disentangling safety from utility.