GuardrailAgent: An Agentic Framework for Jailbreak Defense

Tianze Chang, Junjie Chu, Yixin Wu, Michael Backes, Yang Zhang


Abstract
Despite undergoing safety training to align with human values, large language models (LLMs) remain vulnerable to jailbreak attacks that circumvent built-in safeguards and elicit malicious behaviors given by the adversary. Existing defenses are mostly static and fail to adapt to new or unknown attack strategies, providing limited resilience. To address these, we propose a multi-agent defense framework, GuardrailAgent, that offers adaptive, end-to-end protection. The system autonomously gathers and summarizes attack characteristics into a continuously updated knowledge base that supports both input- and output-side detection. A feedback module integrates misclassified cases for iterative improvement at minimal cost. Tested on over ten representative jailbreaks, the framework achieved a 93.1% input-side detection rate, 93.9% overall detection, and a 94.7% F1 score, while rapidly adapting to unseen attacks. These results demonstrate GuardrailAgent’s outstanding capability to maintain LLM safety in the face of evolving jailbreak tactics.
Anthology ID:
2026.findings-emnlp.70
Volume:
Findings of the Association for Computational Linguistics: EMNLP 2026
Month:
October
Year:
2026
Address:
Budapest, Hungary
Editors:
Hung-Yi Lee, Sunipa Dev, Jessy Li, Desmond Elliott
Venue:
Findings
SIG:
SIGDAT
Publisher:
Association for Computational Linguistics
Note:
Pages:
1412–1428
Language:
URL:
https://aclanthology.org/2026.findings-emnlp.70/
DOI:
Bibkey:
Cite (ACL):
Tianze Chang, Junjie Chu, Yixin Wu, Michael Backes, and Yang Zhang. 2026. GuardrailAgent: An Agentic Framework for Jailbreak Defense. In Findings of the Association for Computational Linguistics: EMNLP 2026, pages 1412–1428, Budapest, Hungary. Association for Computational Linguistics.
Cite (Informal):
GuardrailAgent: An Agentic Framework for Jailbreak Defense (Chang et al., Findings 2026)
Copy Citation:
PDF:
https://aclanthology.org/2026.findings-emnlp.70.pdf