To enhance the multi-turn defense capabilities of Large Language Models, recent studies focus on a “co-evolution framework.” This framework iteratively trains an attacker model to explore vulnerabilities and a target model to learn safety alignment. However, in our preliminary experiments, existing methods suffered from severe training instability. To address this and align with recent optimization methodologies, we introduce GRPO into the training loops of both models. We found that a single alignment cycle sufficiently established the target model’s defense, which in turn stalled the subsequent attacker training. Under the GRPO-based attacker-training pipeline and experimental setting examined in this study, this limitation rendered the iterative co-evolution process ineffective. Furthermore, to maintain model utility, we mix harmful goals with safe goals automatically rewritten from them. In our experiments, mixing safe goals improved utility but yielded higher attack success rates than training without them, revealing a safety-utility trade-off in our setting. Warning: This paper includes examples of harmful language.