Shen Pang
Author directory2026
Fully Unleashing the Multimodal Attacker: Meta-Adaptive Jailbreaking of Vision-Language Models
Shen Pang | Benlei Cui | Yuke Wang | Xuemei Dong | Yuwen Zhai | Jingqun Tang | Haiyang Yu | Hui Xue | Longtao Huang | Haiwen Hong
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Shen Pang | Benlei Cui | Yuke Wang | Xuemei Dong | Yuwen Zhai | Jingqun Tang | Haiyang Yu | Hui Xue | Longtao Huang | Haiwen Hong
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
The safety of large vision-language models is increasingly stress-tested by multimodal jailbreaks, yet existing attacks share one fundamental limitation: the attack itself is *static*. Template-based attacks freeze the image–text layout, while iterative sample-level attacks make only the image–text *content* dynamic, keeping the attack-iteration strategy *fixed* and the attacker’s parameters *frozen*. At the *meta level* they remain static, and we argue this partial dynamism cannot expose the full safety risks of multimodal models.We propose **Meta-Adaptive Multimodal Jailbreaking (MAMJ)**, which lifts adversarial dynamism from the sample level to the meta level by making the attacker’s policy itself the optimisation target along two axes: an **attack strategy prompt (ASP)** 𝜃 in language space, governing how the attack is iterated against a victim VLM, and the **attacker weights** 𝜙, determining the success rate attainable under a given 𝜃.Evolving over a *group of multimodal attack trajectories*, an LLM-based critique of failed trajectories first refines 𝜃, after which group-aggregated attack-success-rate (ASR) rewards update 𝜙. On MM-SafetyBench, MAMJ attains 81.0%, 78.9%, and 82.3% ASR against GPT-4o, Gemini-3-Pro-Preview, and Seed 2.0, exceeding the strongest sample-level baseline by up to 24.1 percentage points, and the learned attacker (𝜃⋆, 𝜙⋆) transfers *without retraining* to unseen victims while remaining effective under representative defences.These results expose a systemic vulnerability of frontier VLMs to meta-adaptive jailbreaks and underscore the need for defences that anticipate meta-level adversaries. Code and checkpoints will be released.