Shuying Ye

Author directory

2026

As large language models (LLMs) are increasingly deployed in real-world applications, ensuring their safety is a critical concern. Most safety evaluations focus on direct harmful requests, overlooking cases where the same harmful intent is disguised through indirect, polite, hypothetical, or contextually framed language. To address this limitation, we propose a pragmatics-based automated red-teaming framework that tests whether LLM safety behavior changes when malicious intent is repackaged through different pragmatic strategies. First, we develop a linguistically grounded taxonomy of seven pragmatic strategies—presupposition, implicature, indirect speech acts, denial disclaimers, politeness, hypothetical framing, and contextual framing—and use it to generate adversarial prompts across seven harm categories. Second, we evaluate these prompts on three Target Models (GPT-4o-mini, Qwen2.5-7B, Llama-3.3-70B), with Claude Haiku as judge and human validation. Our experiments show that pragmatically transformed prompts consistently yield higher Attack Success Rate (ASR) than original seed prompts, with contextual framing emerging as particularly effective. These results suggest LLM safety evaluation should account for pragmatic meaning and inferred intent, not just explicit harmful content.