Jagmohan Chauhan
Author directory2026
Cross-Modal Emergent Misalignment Through Narrow-Domain Fine-Tuning
Han Zhu | Chengkun Cai | Pengcheng Wen | Yanxu Zhu | Jagmohan Chauhan | Lei Li | Yike Guo | Sirui Han
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Han Zhu | Chengkun Cai | Pengcheng Wen | Yanxu Zhu | Jagmohan Chauhan | Lei Li | Yike Guo | Sirui Han
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
The rapid advancement of artificial intelligence has brought safety concerns to the forefront of research. Recent studies indicate that Large Language Models (LLMs) may exhibit emergent misalignment during training progress: when exposed to narrow-domain harmful data, they can develop unsafe behaviors across broad, unrelated tasks. This phenomenon raises a critical question: do current alignment methods genuinely eliminate harmful capabilities, or do they merely condition models to “simulate” safety? To investigate this, we conduct a comprehensive study on cross-modal emergent misalignment involving diverse Multimodal Large Language Models (MLLMs), including vision-language models (VLMs) such as Gemma3 and Qwen2.5-VL and unified any-to-any models including Janus and Bagel. We utilize a multimodal training setup encompassing text-to-text (T2T), text-image-to-text (TI2T), and text-to-image (T2I) tasks to probe safety boundaries. Our experiments yield four critical findings: (1) Cross-modal penetration: Current alignment methods primarily constrain model behavior rather than unlearning harmful knowledge, as evidenced by safety breaches transferring across modalities. (2) Backbone vulnerability: Mechanistic analysis reveals that the LLM backbone, rather than the visual encoder/decoder, is the primary source of cross-modal vulnerabilities. (3) Evaluation gaps: We identify significant deficiencies in current safety evaluation practices, demonstrating that single-modality testing is inadequate for detecting cross-modal risks. These findings underscore the urgent need for comprehensive cross-modal evaluation frameworks to mitigate safety concerns in next-generation multimodal models.