On Large Language Models for Effective Red Teaming

Ming Dong, Hongjing Ren, Bolong Zheng, Chengji Wang, Jingwen Cai, Guanyi Chen, Tingting He


Abstract
Red teaming attacks are a well-established approach for identifying weaknesses in large language models (LLMs). As the generative capabilities of LLMs continue to improve, researchers have increasingly leveraged them to automatically generate red teaming attacks, often by crafting adversarial prompts that target other LLMs. Despite this progress, there is currently no effective strategy for selecting suitable LLMs to serve as red teaming agents. In this work, we propose a systematic framework to investigate how various factors of LLMs influence their effectiveness in generating red teaming attacks, including model security, general capability, and parameter scale. The goal of this study is to understand the mechanisms behind the effectiveness of red teaming LLMs and to provide principled guidance for selecting appropriate red teaming LLM.
Anthology ID:
2026.findings-emnlp.1155
Volume:
Findings of the Association for Computational Linguistics: EMNLP 2026
Month:
October
Year:
2026
Address:
Budapest, Hungary
Editors:
Hung-Yi Lee, Sunipa Dev, Jessy Li, Desmond Elliott
Venue:
Findings
SIG:
SIGDAT
Publisher:
Association for Computational Linguistics
Note:
Pages:
24436–24453
Language:
URL:
https://aclanthology.org/2026.findings-emnlp.1155/
DOI:
Bibkey:
Cite (ACL):
Ming Dong, Hongjing Ren, Bolong Zheng, Chengji Wang, Jingwen Cai, Guanyi Chen, and Tingting He. 2026. On Large Language Models for Effective Red Teaming. In Findings of the Association for Computational Linguistics: EMNLP 2026, pages 24436–24453, Budapest, Hungary. Association for Computational Linguistics.
Cite (Informal):
On Large Language Models for Effective Red Teaming (Dong et al., Findings 2026)
Copy Citation:
PDF:
https://aclanthology.org/2026.findings-emnlp.1155.pdf