Despite advances in safety alignment, prompt-rewriting attacks such as persona modulation, fictional framing and persuasion-based reformulation, can bypass safety filters even on frontier models. Existing defenses either rely on non-scalable human curation or white-box optimisation that overfits to specific model internals, leaving aligned models brittle against the very class of adaptive black-box adversaries they will face in deployment. To address this gap, we introduce CHASE (Co-evolutionary Hardening through Adversarial Safety-Escalation), a closed-loop red-blue teaming framework in which a black-box attacker and a safety-aligned defender co-evolve. The attacker is trained via Group Relative Policy Optimization (GRPO) under a multiplicative reward that jointly enforces bypass effectiveness and intent fidelity, while the defender is hardened on the harvested adversarial rewrites through a two-stage GRPO + rejection-sampled SFT pipeline balanced with benign data. Evaluated on BeaverTails and JailbreakBench against five held-out attack families (PAIR, TAP, AutoDAN, PAP, Translation), CHASE cuts mean StrongREJECT score by 43.2% on Llama-3.1-8B-Instruct with 0% false refusal on benign (Alpaca) prompts, at a measured helpfulness cost of a 55.6% refusal rate on XSTest safe prompts and a 1.92-point MT-Bench reduction. A controlled ablation that varies only the training attack distribution shows that this generalisation comes from the template-free attack distribution rather than the pipeline, and the result replicates on Qwen2.5-7B-Instruct. Across the two similarly sized open models we study, these results suggest that template-free RL exploration recovers latent attack primitives that transfer across mechanistically distinct attack families.