Fixed-target backdoor attacks can silently hijack the outputs of large language models by driving models to generate attacker-specified sequences upon trigger activation, motivating lightweight inference-time detection. Recent defenses like ConfGuard, which we term Local-window Confidence Detectors, leverage sliding windows to identify local spans where the top-1 probability, referred to as token confidence, exceeds 0.99 as indicators of backdoor activation. However, benign deterministic tasks such as math reasoning can exhibit similar sustained high-confidence patterns, causing confidence conflation. On the GSM8K dataset, ConfGuard’s default settings yield a 72.89% false positive rate, causing substantial over-defense. Our empirical analysis reveals that benign deterministic generation exhibits sparse confidence "dips", whereas backdoor-triggered generation forms a sustained high-confidence "plateau". Building on this, we propose RatioGuard, which utilizes a global high-confidence token ratio computed over the generated prefix to address confidence conflation. Across models and datasets, RatioGuard reduces false positives by up to 71.57 percentage points over ConfGuard while maintaining over 99% true positive rate with 1.005× runtime latency.

As large language models (LLMs) increasingly rely on web-scale corpora, membership inference attacks (MIAs) have become essential tools for auditing unauthorized data usage. However, traditional token-level MIAs fail to generalize to large reasoning models (LRMs) due to data distribution shifts introduced during fine-tuning. While recent studies address this by encoding reasoning traces into latent semantic spaces, they sacrifice the interpretability strictly required for real-world copyright verification. In this paper, we demonstrate that an LRM’s internal memory states are explicitly leaked through its stylistic word choices. Our analysis reveals a distinct lexical dichotomy: non-member traces tend toward a "guessing-like" tone, whereas member traces exhibit a "remember-like" style. Motivated by this interesting phenomenon, we propose TraceMIA, an interpretable MIA framework that audits training corpora by computing membership scores based on the relative frequencies of discriminative cue words. Extensive empirical evaluations across representative LRMs and diverse datasets demonstrate that TraceMIA achieves robust detection efficacy while providing the transparent, human-readable evidence critical for practical auditing scenarios.

The rapid development of large language models (LLMs) has created an urgent need for identifying machine-generated texts, and text watermarking technology has proven to be an effective solution. However, current watermarking methods, while demonstrating strong detectability, significantly degrade text quality due to the introduction of unnatural tokens. The main reason lies in the fact that these methods ignore the importance of semantic information in the watermarking process. To address this issue, we note that the logit vector produced by LLMs encodes both semantic understanding of input texts and prediction confidence across different tokens. Therefore, we propose a novel Semantic Self-Guided Watermarking (SSGW) framework that leverages the LLM itself to generate a guidance logic vector that assists in watermarking while producing the original one concurrently. Subsequently, we design a transform module to analyze these two vectors comprehensively and then transform them into adaptive watermark logits for different candidate tokens, thereby reducing the possibility of selecting inappropriate tokens. Experimental results confirm the effectiveness of our method in achieving superior performance in both watermark detectability and text quality preservation. The source code will be made publicly available upon acceptance.

Although large reasoning models (LRMs) exhibit exceptional mathematical reasoning capabilities on clean inputs, their reasoning accuracy drops substantially in the presence of character-level noise such as typographical errors. Critically, their confidence estimates fail to reflect the corresponding decline in reasoning accuracy. While confidence calibration offers a principled solution, existing methods predominantly target clean inputs, leaving noisy scenarios largely unexplored. To address this gap, we propose DisCal (Distribution-aware Calibration), a confidence calibration framework for character-level noisy inputs. DisCal extracts uncertainty signals from both the empirical answer distribution and the model’s predictive distribution, and integrates them via a learned calibrator to produce well-calibrated confidence. Experiments across multiple mathematical reasoning benchmarks demonstrate that DisCal consistently outperforms existing calibration methods under noisy inputs, reducing Expected Calibration Error (ECE) by up to 39.21% and improving Area Under the Receiver Operating Characteristic Curve (AUROC) by up to 31.44%.

Large language models (LLMs) have achieved groundbreaking progress in Natural Language Processing (NLP). Despite the numerous advantages of LLMs, they also pose significant safety risks. Self-evaluation mechanisms have gained increasing attention as a key safeguard to ensure safe and controllable content generation. However, LLMs often exhibit overconfidence, which seriously compromises the accuracy of safety self-evaluation. To address this challenge, we propose SafeConf, a method to enhance the safety self-evaluation capability of LLMs through confidence calibration. The method performs semantic mutations on the original safety evaluation questions and adopts a self-consistency strategy to quantify confidence based on answer accuracy on the mutated questions. Finally, these confidence scores are used to construct a dataset for fine-tuning. We conducte experiments on both Chinese and English datasets. The results show that SafeConf improves self-evaluation accuracy by an average of 5.86% and 7.79% over the state-of-the-art baseline methods on Qwen2.5-7B-Instruct and Llama3-8B-Instruct models, respectively, without affecting the general capabilities of the models.

Large language models (LLMs) have achieved significant advances but can potentially generate harmful content such as social biases, extremism, and misinformation. Red teaming is a promising approach to enhance model safety by creating adversarial prompts to test and improve model robustness. However, existing red-teaming methods often require expensive fine-tuning, especially for large LLMs. We propose the Dynamic Evil Score-Guided Decoding framework (DESGD), an efficient red-teaming method that does not increase computational cost with the target model size. DESGD introduces the concept of an ‘evil score’ to dynamically evaluate the potential of tokens to contribute to harmful outputs during decoding. This framework constructs a small unsafe model using an adversarial dataset and adjusts the logits vector of the target model based on the evil score. Experiments show that DESGD achieves an ASR of 92.83% on the Llama-3.2-3B-Instruct model, compared to 83.48% with adversarial fine-tuning while using less computational resources. Similarly, on the Qwen2.5-3B-Instruct model, DESGD reaches an ASR of 88.62%, outperforming adversarial fine-tuning (77.56%).