CHASE: Adversarial Red-Blue Teaming for Improving LLM Safety using Reinforcement Learning

Rahul Markasserithodi, Aditya Joshi, Yuekang Li, Ishmanbir Singh, Chris Yoo, Alan Niu

Abstract
Despite advances in safety alignment, prompt-rewriting attacks such as persona modulation, fictional framing and persuasion-based reformulation, can bypass safety filters even on frontier models. Existing defenses either rely on non-scalable human curation or white-box optimisation that overfits to specific model internals, leaving aligned models brittle against the very class of adaptive black-box adversaries they will face in deployment. To address this gap, we introduce CHASE (Co-evolutionary Hardening through Adversarial Safety-Escalation), a closed-loop red-blue teaming framework in which a black-box attacker and a safety-aligned defender co-evolve. The attacker is trained via Group Relative Policy Optimization (GRPO) under a multiplicative reward that jointly enforces bypass effectiveness and intent fidelity, while the defender is hardened on the harvested adversarial rewrites through a two-stage GRPO + rejection-sampled SFT pipeline balanced with benign data. Evaluated on BeaverTails and JailbreakBench against five held-out attack families (PAIR, TAP, AutoDAN, PAP, Translation), CHASE cuts mean StrongREJECT score by 43.2% on Llama-3.1-8B-Instruct with 0% false refusal on benign (Alpaca) prompts, at a measured helpfulness cost of a 55.6% refusal rate on XSTest safe prompts and a 1.92-point MT-Bench reduction. A controlled ablation that varies only the training attack distribution shows that this generalisation comes from the template-free attack distribution rather than the pipeline, and the result replicates on Qwen2.5-7B-Instruct. Across the two similarly sized open models we study, these results suggest that template-free RL exploration recovers latent attack primitives that transfer across mechanistically distinct attack families.
Anthology ID:
2026.aacl-long.36
Volume:
Proceedings of the 15th International Joint Conference on Natural Language Processing and the 5th Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics (Volume 1: Long Papers)
Month:
November
Year:
2026
Address:
Hengqin, China
Editors:
Yue Zhang, Ruifeng Xu, Ayu Purwarianti, Tong Rong, Jun Suzuki
Venues:
AACL | IJCNLP
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
742–758
Language:
URL:
https://aclanthology.org/2026.aacl-long.36/
DOI:
Bibkey:
Cite (ACL):
Rahul Markasserithodi, Aditya Joshi, Yuekang Li, Ishmanbir Singh, Chris Yoo, and Alan Niu. 2026. CHASE: Adversarial Red-Blue Teaming for Improving LLM Safety using Reinforcement Learning. In Proceedings of the 15th International Joint Conference on Natural Language Processing and the 5th Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics (Volume 1: Long Papers), pages 742–758, Hengqin, China. Association for Computational Linguistics.
Cite (Informal):
CHASE: Adversarial Red-Blue Teaming for Improving LLM Safety using Reinforcement Learning (Markasserithodi et al., AACL-IJCNLP 2026)
Copy Citation:
PDF:
https://aclanthology.org/2026.aacl-long.36.pdf
PDF Cite Search Fix data