With the rapid development of Large Language Models (LLMs), research related to LLM-based persuasion has become a hotspot in the field. However, existing studies mostly focus on enhancing LLM agents’ persuasive capabilities, while generally neglecting the issue of their vulnerability to persuasion. In this paper, we analyse the root causes of LLM agents’ vulnerability to persuasion and find that it mainly stems from two contradictions and limitations. First, the purpose of catering to human preferences in pre-training phase conflicts with the need to resist manipulation in persuasion scenarios. Second, LLMs are often restricted to play a helpful assistant during training, and the singularity of this personality setting makes it unusually easy to develop targeted persuasion techniques for them. To eliminate LLMs’ vulnerability to persuasion, we propose the Backfire-R1 framework, which helps LLMs build persuasion resistance and better align with user objective by enabling them to learn the reasoning logic of humans with different personalities in persuasion scenarios. Comprehensive experimental results show that, the proposed method significantly improves the persuasion robustness of LLM agents, reducing the opinion shift score by 62.36% while not impairing the model’s general capabilities. Code is now available at: https://github.com/super-jw/Backfire-R1.

With the widespread application of large language models (LLMs) across various domains, techniques for enhancing their security have progressed rapidly. In this paper, we reveal that although existing defense methods can improve the robustness of LLMs against jailbreaks, they compromise usability, i.e., reducing general capabilities or causing the over-refusal problem. From the perspective of LLM mechanism interpretability, we discover that these methods fail to establish a boundary that exactly distinguishes safe and harmful feature representations. Therefore, boundary-safe representations close to harmful representations are inevitably disrupted, leading to a decline in usability. To address this issue, we propose X-Boundary to push harmful representations away from boundary-safe representations and obtain an exact distinction boundary. In this way, harmful representations can be precisely erased without disrupting safe ones. Experimental results show that X-Boundary achieves state-of-the-art defense performance against both single-turn and multi-turn jailbreak attacks, while reducing the over-refusal rate by about 20% and maintaining nearly complete general capability. Furthermore, we theoretically prove and empirically verify that X-Boundary can accelerate the convergence process during training.

With the widespread of Large Language Models (LLMs), there has been an increasing need to detect LLM-generated texts, prompting extensive research in this area. However, existing detection methods mainly evaluate on static benchmarks, which neglect the evolving nature of LLMs. Relying on existing static benchmarks could create a misleading sense of security, overestimating the real-world effectiveness of detection methods.To bridge this gap, we introduce EvoBench, a dynamic benchmark considering a new dimension of generalization across continuously evolving LLMs.EvoBench categorizes the evolving LLMs into (1) updates over time and (2) developments like finetuning and pruning, covering 7 LLM families and their 29 evolving versions. To measure the generalization across evolving LLMs, we introduce a new EMG (Evolving Model Generalization) metric. Our evaluation of 14 detection methods on EvoBench reveals that they all struggle to maintain generalization when confronted with evolving LLMs. To mitigate the generalization problems, we further propose improvement strategies. For zero-shot detectors, we propose pruning the scoring model to extract shared features. For supervised detectors, we also propose a practical training strategy.Our research sheds light on critical challenges in real-world LLM-generated text detection and represents a significant step toward practical applications.