Despite undergoing safety training to align with human values, large language models (LLMs) remain vulnerable to jailbreak attacks that circumvent built-in safeguards and elicit malicious behaviors given by the adversary. Existing defenses are mostly static and fail to adapt to new or unknown attack strategies, providing limited resilience. To address these, we propose a multi-agent defense framework, GuardrailAgent, that offers adaptive, end-to-end protection. The system autonomously gathers and summarizes attack characteristics into a continuously updated knowledge base that supports both input- and output-side detection. A feedback module integrates misclassified cases for iterative improvement at minimal cost. Tested on over ten representative jailbreaks, the framework achieved a 93.1% input-side detection rate, 93.9% overall detection, and a 94.7% F1 score, while rapidly adapting to unseen attacks. These results demonstrate GuardrailAgent’s outstanding capability to maintain LLM safety in the face of evolving jailbreak tactics.

Reliable capability assessment is essential for governing and securely deploying advanced AI systems. However, "AI Sandbagging," where models strategically suppress their true capabilities during evaluation, undermines existing assessment frameworks by making it difficult to distinguish genuine limitations from strategic concealment. To address this challenge, we propose a new black-box sandbag detection method inspired by "honesty inertia," a psychological phenomenon where humans struggle to maintain deception after answering simple questions truthfully. We hypothesize that large language models exhibit an analogous effect: inserting several simple, unrelated questions, which target models should answer correctly, before the target query induces honest inertia, causing sandbag models to involuntarily increase accuracy while standard models remain stable. Through extensive experiments on multiple datasets with various model architectures, we confirm the validity of this hypothesis and establish this differential response as a reliable, model-independent sandbag detection signal. We further provide interpretability support for this phenomenon from two dimensions: theoretical derivation and hidden state analysis of models. This work offers a practical tool for enhancing AI audit and supervision.

In-the-real-world datasets provide a crucial reflection of real-world adversarial distributions and hold significant practical importance for evaluating the safety of T2I models. However, existing in-the-real-world unsafe benchmarks are not tailored for T2I generation: many prompts lack visual specificity or concrete scenarios, producing distorted and low-fidelity images that are misclassified as safe by automated detectors. This acts as a confounder of successful safety alignment and compromises the accuracy of safety measurement. To address this, we propose RW2Unsafe, an automated three-module framework that reformulates source in-the-real-world posts into visually descriptive T2I prompts while preserving the original harmful intent. We use it to construct rw_t2i_4chan, a benchmark of 2,000 high-quality in-the-real-world unsafe T2I prompts. Auditing five advanced T2I models, we find that prior approaches relying on source prompts systematically underestimate safety risks: on Bagel, for instance, the mean unsafe score under source prompts is under one-third of that under rewritten prompts. We hope the proposed framework and dataset could contribute to more accurate safety measurement and safer T2I models.

Accurately assessing jailbreak success remains unsolved, as existing automatic methods often diverge from human perception. To address this, we propose JADES, a universal evaluation framework that decomposes a harmful query into weighted sub-questions, scores each sub-answer, and aggregates the scores into a final decision; an optional fact-checking module further mitigates hallucination-induced misjudgments. We validate JADES on JailbreakQR, a meticulously annotated benchmark of 400 prompt–response pairs. Under binary settings (success/failure), JADES achieves 98.5% human agreement, outperforming strong baselines by over 9%. Re-evaluating five popular attacks across seven LLMs reveals systematic overestimation of jailbreak. JADES also enables granular ternary evaluation (failure/partial success/success), uncovers that partial successes dominate current reported successes, and recalibrates the perceived severity of existing attacks. Overall, JADES reveals a major limitation of current jailbreak evaluation and, together with ternary assessment, enables a more accurate understanding of the true jailbreak effectiveness.

Jailbreak attacks aim to bypass the LLMs’ safeguards. While researchers have proposed different jailbreak attacks in depth, they have done so in isolation—either with unaligned settings or comparing a limited range of methods. To fill this gap, we present a large-scale evaluation of various jailbreak attacks. We collect 17 representative jailbreak attacks, summarize their features, and establish a novel jailbreak attack taxonomy. Then we conduct comprehensive measurement and ablation studies across nine aligned LLMs on 160 forbidden questions from 16 violation categories. Also, we test jailbreak attacks under eight advanced defenses. Based on our taxonomy and experiments, we identify some important patterns, such as heuristic-based attacks, which could achieve high attack success rates but are easy to mitigate by defenses. Our study offers valuable insights for future research on jailbreak attacks and defenses and serves as a benchmark tool for researchers and practitioners to evaluate them effectively.

Significant advancements have recently been made in large language models, represented by GPT models.Users frequently have multi-round private conversations with cloud-hosted GPT models for task optimization.Yet, this operational paradigm introduces additional attack surfaces, particularly in custom GPTs and hijacked chat sessions.In this paper, we introduce a straightforward yet potent Conversation Reconstruction Attack.This attack targets the contents of previous conversations between GPT models and benign users, i.e., the benign users’ input contents during their interaction with GPT models.The adversary could induce GPT models to leak such contents by querying them with designed malicious prompts.Our comprehensive examination of privacy risks during the interactions with GPT models under this attack reveals GPT-4’s considerable resilience.We present two advanced attacks targeting improved reconstruction of past conversations, demonstrating significant privacy leakage across all models under these advanced techniques.Evaluating various defense mechanisms, we find them ineffective against these attacks.Our findings highlight the ease with which privacy can be compromised in interactions with GPT models, urging the community to safeguard against potential abuses of these models’ capabilities.