Wei Qian
Author directory2026
Towards Understanding the Robustness of Self-Consuming Generative Models
Aobo Chen | Wei Qian | Chenglin Miao | Mengdi Huai
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Aobo Chen | Wei Qian | Chenglin Miao | Mengdi Huai
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Recent advances in deep generative models have made it increasingly difficult to distinguish real-world data from model-generated content. Consequently, successive model generations are trained on synthetic data from prior models, forming a self-consuming training loop. Despite being powerful, it also opens a new attack surface, i.e., test-time poisoning attacks, adversaries exploit model vulnerabilities during the inference stage to generate malicious test-time outputs that are subsequently incorporated into future training data. In this paper, for the first time, we propose a new class of malicious test-time poisoning attack against self-consuming generative methods. However, uncovering the failure modes of self-consuming generative models under test-time poisoning attacks is highly challenging due to the need to simultaneously maintain stealthy data manipulations while achieving fine-grained control over the activation and propagation of malicious behaviors across self-consuming training iterations. To address these challenges, our attacks provide fine-grained control and introduce a novel stealth paradigm that achieves both effectiveness and stealthiness by camouflaging stealthy malicious data within the training process and selectively activating concealed attack behaviors at targeted later stages. Notably, our attacks are compatible with various adversarial attack generating algorithms and self-consuming algorithms. Extensive experiments under different settings, including black-box scenarios, demonstrate that self-consuming generative models are broadly vulnerable to the proposed test-time poisoning attacks.