Fixed-target backdoor attacks can silently hijack the outputs of large language models by driving models to generate attacker-specified sequences upon trigger activation, motivating lightweight inference-time detection. Recent defenses like ConfGuard, which we term Local-window Confidence Detectors, leverage sliding windows to identify local spans where the top-1 probability, referred to as token confidence, exceeds 0.99 as indicators of backdoor activation. However, benign deterministic tasks such as math reasoning can exhibit similar sustained high-confidence patterns, causing confidence conflation. On the GSM8K dataset, ConfGuard’s default settings yield a 72.89% false positive rate, causing substantial over-defense. Our empirical analysis reveals that benign deterministic generation exhibits sparse confidence "dips", whereas backdoor-triggered generation forms a sustained high-confidence "plateau". Building on this, we propose RatioGuard, which utilizes a global high-confidence token ratio computed over the generated prefix to address confidence conflation. Across models and datasets, RatioGuard reduces false positives by up to 71.57 percentage points over ConfGuard while maintaining over 99% true positive rate with 1.005× runtime latency.

Large Language Models (LLMs) have been widely applied in various domains such as education and healthcare, making safety assurance crucial. Jailbreak attacks, a method used in red-teaming, can help evaluate and improve the defensive strategies of LLMs. However, existing jailbreak methods often overlook the semantic differences across categories of harmful questions, leading to inconsistent success rates and reduced overall attack effectiveness. We propose the first category-aware jailbreak framework, SHARP, which incorporates the semantic category of harmful questions into prompt generation. Trained on a verified jailbreak dataset, SHARP enables the model to learn category-specific semantic features and adaptively generate prompts that bypass safety mechanisms. The method combines two-stage LoRA fine-tuning, and DPO-based reinforcement learning to optimize both attack success and category alignment. Experiments show that SHARP significantly improves attack success rates and achieves better cross-category robustness compared to the state-of-the-art (SOTA) baselines, providing an efficient and scalable tool for evaluating LLM safety.