Barak Chizi

Author directory

2026

Red-teaming has become a standard tool for probing alignment and safety weaknesses in large language models, but most existing methods treat each attack session in isolation and discard what was learned once the dialogue ends. We introduce **LongMem-RAT**, a multi-agent red-teaming framework equipped with long-term memory that accumulates, consolidates, and reuses insights from past attacks across sessions. This design enables cross-session generalization, reduces redundant exploration, and focuses new attacks on fragile alignment behaviors. Across multiple proprietary and open-weight models, LongMem-RAT improves Attack Success Rate (ASR) over a fully reproduced X-Teaming baseline by **21.73 percentage points on HarmBench and 18.40 points on AdvBench** (20.06 points on average). Memory ablations show that long-term memory improves ASR by 13.23 and 6.07 points and reduces Average Turns to Success (ATtS) by 0.37 and 1.92 turns on HarmBench and AdvBench, respectively, averaging a 9.65-point ASR gain and a 1.14-turn ATtS reduction.