Zhongzhi Li
Author directory2026
Beyond Prompts: Discovering Jailbreak Motifs in Sparse Feature Space
Zhongzhi Li | Jiahui Han | Jialing Tao | Huiqi Deng | Na Zou | Ninghao Liu
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Zhongzhi Li | Jiahui Han | Jialing Tao | Huiqi Deng | Na Zou | Ninghao Liu
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Large language models (LLMs) remain vulnerable to jailbreak attacks that exhibit diverse prompts and rapidly evolving strategies. Existing defenses often operate at the level of prompts or predefined attack methods, limiting their ability to generalize to unseen attacks that share similar underlying structure but differ in wording. In this paper, we investigate whether seemingly different jailbreak attacks exhibit shared SAE feature co-activation patterns, and propose a motif-based framework that models attacks through these recurring patterns. Specifically, we first extract sparse autoencoder (SAE) features from jailbreak queries and discover motifs as recurring co-activation patterns via nonnegative matrix factorization. We show that motifs capture stable relationships across attack methods and generalize to unseen attacks. Building on this structure, we synthesize new jailbreak queries by composing motif-guided attack patterns. We then use the generated queries to construct preference-training data for Direct Preference Optimization (DPO). Experiment results show that SAE motifs serve as a stable and transferable interface for understanding and detecting jailbreak attacks, as well as enabling effective defenses that generalize beyond fixed prompt templates. Training target models with motif-guided synthetic data improves robustness to unseen attacks with modest changes in utility. Our work provides a practical methodology for advancing data-centric LLM safety through structured modeling of jailbreak attacks.
2025
CMMaTH: A Chinese Multi-modal Math Skill Evaluation Benchmark for Foundation Models
Zhongzhi Li | Ming-Liang Zhang | Pei-Jie Wang | Jian Xu | Rui-Song Zhang | Yin Fei | Zhi-Long Ji | Jin-Feng Bai | Zhen-Ru Pan | Jiaxin Zhang | Cheng-Lin Liu
Proceedings of the 31st International Conference on Computational Linguistics
Zhongzhi Li | Ming-Liang Zhang | Pei-Jie Wang | Jian Xu | Rui-Song Zhang | Yin Fei | Zhi-Long Ji | Jin-Feng Bai | Zhen-Ru Pan | Jiaxin Zhang | Cheng-Lin Liu
Proceedings of the 31st International Conference on Computational Linguistics
With the rapid advancements in multimodal large language models, evaluating their multimodal mathematical capabilities continues to receive wide attention. Although datasets such as MathVista have been introduced for evaluating mathematical capabilities in multimodal scenarios, there remains a lack of evaluation tools and datasets tailored for fine-grained assessment in Chinese K12 education. To systematically evaluate the ability of multimodal large models to solve Chinese multimodal mathematical problems, we propose a Chinese Multi-modal Math Skill Evaluation Benchmark (CMMaTH), containing 23,856 multimodal K12 math related questions, making it the largest Chinese multimodal mathematical problem benchmark to date. CMMaTH includes questions ranging from elementary to high school levels, offering greater diversity in problem types, solution goals, visual elements, detailed knowledge points, and standard solution annotations. To facilitate stable, fast, and cost-free model evaluation, we have developed an open-source tool called GradeGPT, which is integrated with the CMMaTH dataset. Our data and code are available at https://github.com/zzli2022/CMMaTH.