Large Vision-Language Models (LVLMs) have demonstrated remarkable multimodal understanding capabilities, yet their inherent security vulnerabilities remain a pressing concern. Prior studies have indicated that enhancing chain-of-thought reasoning can inadvertently facilitate jailbreaks; however, existing attempts often rely on image masking, which only modestly extends reasoning depth and performs poorly against advanced commercial closed-source models. To bridge this gap, we propose an effective jailbreak method, Text–Image Segmentation and Reassembly (TISR). TISR disperses harmful semantic fragments across the visual modality while employing text-based role-playing to guide their reassembly. This strategic organization conceals surface-level toxicity while preserving malicious intent, thereby extending LVLM reasoning chains and enabling successful jailbreaks. Our experiments show that LVLMs, leveraging their visual understanding and reasoning abilities, can reconstruct complete harmful semantics while failing to uphold safety constraints. Extensive evaluations across massive LVLMs demonstrate that the proposed TISR achieves state-of-the-art attack performance, surpassing prior multi-modal jailbreak methods. Specifically, it attains average Attack Success Rates (ASR) exceeding 90% across three benchmark datasets on four leading commercial LVLMs.

Understanding the vulnerabilities of Large Vision Language Models (LVLMs) to jailbreak attacks is essential for their responsible real-world deployment. Most previous work requires access to model gradients, or is based on human knowledge (prompt engineering) to complete jailbreak, and they hardly consider the interaction of images and text, resulting in inability to jailbreak in black box scenarios or poor performance. To overcome these limitations, we propose a Prior-Guided Bimodal Interactive Black-Box Jailbreak Attack for toxicity maximization, referred to as PBI-Attack. Our method begins by extracting malicious features from a harmful corpus using an alternative LVLM and embedding these features into a benign image as prior information. Subsequently, we enhance these features through bidirectional cross-modal interaction optimization, which iteratively optimizes the bimodal perturbations in an alternating manner through greedy search, aiming to maximize the toxicity of the generated response. The toxicity level is quantified using a well-trained evaluation model.Experiments demonstrate that PBI-Attack outperforms previous state-of-the-art jailbreak methods, achieving an average attack success rate of 92.5% across three open-source LVLMs and around 67.3% on three closed-source LVLMs.redDisclaimer: This paper contains potentially disturbing and offensive content.

Understanding the vulnerabilities of Large Vision Language Models (LVLMs) to jailbreak attacks is essential for their responsible real-world deployment. Most previous work requires access to model gradients, or is based on human knowledge (prompt engineering) to complete jailbreak, and they hardly consider the interaction of images and text, resulting in inability to jailbreak in black box scenarios or poor performance. To overcome these limitations, we propose a Prior-Guided Bimodal Interactive Black-Box Jailbreak Attack for toxicity maximization, referred to as PBI-Attack. Our method begins by extracting malicious features from a harmful corpus using an alternative LVLM and embedding these features into a benign image as prior information. Subsequently, we enhance these features through bidirectional cross-modal interaction optimization, which iteratively optimizes the bimodal perturbations in an alternating manner through greedy search, aiming to maximize the toxicity of the generated response. The toxicity level is quantified using a well-trained evaluation model. Experiments demonstrate that PBI-Attack outperforms previous state-of-the-art jailbreak methods, achieving an average attack success rate of 92.5% across three open-source LVLMs and around 67.3% on three closed-source LVLMs. Disclaimer: This paper contains potentially disturbing and offensive content.