Cheng-zhong Xu
Author directory2026
Seeing the Poem: Image-Semantic Detection of AI-Generated Modern Chinese Poetry with MLLMs
Shanshan Wang | Fengying Ye | Hanjia Lyu | Caiwen Gou | Junchao Wu | Jingming Yao | Cheng-zhong Xu | Jiebo Luo | Derek F. Wong
Findings of the Association for Computational Linguistics: EMNLP 2026
Shanshan Wang | Fengying Ye | Hanjia Lyu | Caiwen Gou | Junchao Wu | Jingming Yao | Cheng-zhong Xu | Jiebo Luo | Derek F. Wong
Findings of the Association for Computational Linguistics: EMNLP 2026
Previous detection studies have shown that LLMs cannot be effectively used as detectors, but these studies have not addressed modern Chinese poetry. Moreover, no relevant research has explored the performance of LLMs in detecting modern Chinese poetry. This paper evaluates and enhances the performance of LLMs as detectors for modern Chinese poetry, and proposes an image-semantic guided poetry detection method. Compared with traditional detection approaches, our method innovatively incorporates images that reflect the content of the poetry. Through example-driven approaches, our method effectively integrates information such as meaning, imagery, and feeling from the image, then forms a complementary judgment with the poem text. Experimental results demonstrate that the LLM detectors based on our method outperform baseline detectors based on plain text, and even surpass the best-performing traditional detector, RoBERTa. The Gemini detector using our method achieves a Macro-F1 score of 85.65%, reaching the state-of-the-art level. The performance improvements of different LLM detectors on multiple LLMs-generated data prove the effectiveness of our method.
GPC-E4: Generalized Consistency and Entropic Evidence for Backdoor Detection in Vision Language Models
Xuan Wang | Xuehu Yan | Yuliang Lu | Kejiang Ye | Cheng-zhong Xu | Xitong Gao
Findings of the Association for Computational Linguistics: IJCNLP-AACL 2026
Xuan Wang | Xuehu Yan | Yuliang Lu | Kejiang Ye | Cheng-zhong Xu | Xitong Gao
Findings of the Association for Computational Linguistics: IJCNLP-AACL 2026
Vision-language models (VLMs) are increasingly used through black-box, open-ended interfaces, where input-level backdoor screening must work without model parameters, gradients, logits, or training data. We present GPC-E4, a detector for single image-prompt queries that probes a VLM with mild, semantics-preserving visual and textual perturbations. The method measures whether the output behavior contracts unusually under these probes: GPC captures dominant-mode or semantic consistency, while E4 summarizes predictive entropy, augmentation mutual information, minimal collapse, and local slope along perturbation paths. A fixed robust normalization combines these signals and can operate without a known-clean calibration set. Across COCO, Flickr30k, Caltech101, and MMBench-style evaluations with LLaVA-1.5, MiniGPT-4, CLIP, CoOp, and Qwen3-VL, GPC-E4 outperforms STRIP, SCALE, TeCo, and BDet in both closed-set and open-ended settings. Additional analyses under alternative stressors, prompt and distribution shifts, zero clean calibration, VLM-native attacks, and detector-aware attacks characterize when the contraction signal is reliable and when it weakens.
Emergent Backdoors in Agentic Recursive Training
Xuan Wang | Yuliang Lu | Kejiang Ye | Cheng-zhong Xu | Xitong Gao
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Xuan Wang | Yuliang Lu | Kejiang Ye | Cheng-zhong Xu | Xitong Gao
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Agentic recursive training systems increasingly reuse model-generated tasks, proxy feedback, and selected trajectories to refine policies over multiple rounds. This creates feedback loops in which small conditional biases can accumulate across training and alter later selection and updates. We study whether trigger-conditional proxy-feedback biases can produce emergent backdoors in a controlled recursive training loop. A Curriculum Agent proposes tasks, an Executor Agent generates rollouts, and environment feedback is selectively biased on rare markers, affecting reward, selection, and curriculum updates. Under matched biased supervision, Reward Hacking + SFT and Reward Hacking + RLHF reach 31.27% and 53.85% ASR, while the full loop reaches 97.62% at comparable clean accuracy. The effect persists across Qwen3-4B, LLaMA-3-8B, BLIP-2-OPT-2.7B, and Qwen2.5-14B, transfers to GSM8K, BBH, and HotpotQA, extends to a tool-augmented setting, and is only partially reduced by post-hoc defenses. These results suggest that feedback and curriculum mechanisms in agentic recursive training should be audited for long-horizon bias amplification.
Towards a Comprehensive Scaling Law of Mixture-of-Experts
Guoliang Zhao | Yuhan Fu | Shuaipeng Li | Samm Sun | Ruobing Xie | An Wang | Weidong Han | Zhen Yang | Weixuan Sun | Yudong Zhang | Cheng-zhong Xu | Di Wang | Jie Jiang
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Guoliang Zhao | Yuhan Fu | Shuaipeng Li | Samm Sun | Ruobing Xie | An Wang | Weidong Han | Zhen Yang | Weixuan Sun | Yudong Zhang | Cheng-zhong Xu | Di Wang | Jie Jiang
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Mixture-of-Experts (MoE) has emerged as the dominant paradigm for parameter-efficient scaling and cost-effective deployment of Large Language Models (LLMs). However, scaling laws developed for dense models are fundamentally inapplicable to MoE architectures, owing to three critical challenges: the multiplicity of governing factors, their intricate interdependencies, and the non-monotonic nature of their effects on model performance. These challenges collectively demand a rigorous and fine-grained investigation into MoE-specific scaling laws. In this work, we present a systematic decomposition of MoE configurations, identifying five key factors that influence model performance from both size and structural perspectives: data size (D), total model size (N), activated model size (Na), number of active experts (G), and ratio of shared experts (S). We conduct 450 controlled experiments to characterize the marginal effects of each factor, culminating in a comprehensive and precise joint MoE scaling law. Building on this foundation, we derive the theoretically optimal and practically efficiency-aware optimal configurations for G, S and Na/N with detailed analyses. Our findings reveal that the optimal settings for G and S are invariant to model and data size, while the optimal activation ratio Na/N grows sparser as N increases. Our proposed MoE scaling law provides accurate and insightful guidance for future MoE model design and training.
A2M: Trace-Optimized Agent Hijacking in the MCP Ecosystem
Laizhen Li | Xuan Wang | Peicheng Zhao | Juanjuan Zhao | Kejiang Ye | Cheng-zhong Xu | Xitong Gao
Proceedings of the 15th International Joint Conference on Natural Language Processing and the 5th Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics (Volume 1: Long Papers)
Laizhen Li | Xuan Wang | Peicheng Zhao | Juanjuan Zhao | Kejiang Ye | Cheng-zhong Xu | Xitong Gao
Proceedings of the 15th International Joint Conference on Natural Language Processing and the 5th Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics (Volume 1: Long Papers)
Agents using the Model Context Protocol (MCP) rely on semantic matching to select tools from third-party servers, exposing a semantic supply-chain risk through attacker-controlled metadata and outputs. We introduce A2M (Attraction-to-Manipulation), a two-stage black-box framework for hijacking MCP agents. The Attraction phase optimizes tool metadata to increase invocation probability; the Manipulation phase uses execution traces to refine adversarial tool returns that steer agents toward attacker-desired outcomes. On LiveMCPBench, direct attacks optimized and evaluated on GLM-4.6 achieve a macro-average malicious tool invocation rate of 93.6% across four scenarios, increase weighted token costs to 32.4× the benign baseline under Cognitive Denial of Service, and attain a mean attack success rate of 74.4% across Information Exfiltration, Environment Integrity Compromise, and Reasoning Derailment. Transfer to four other models without re-optimization yields corresponding macro-averages of 63.6%, 2.7×, and 24.5%. These findings motivate stronger tool vetting and runtime isolation in MCP ecosystems. Code is publicly available at https://github.com/Lilaizhen/A2M.
2025
Mixture of Weight-shared Heterogeneous Group Attention Experts for Dynamic Token-wise KV Optimization
Guanghui Song | Dongping Liao | Yiren Zhao | Kejiang Ye | Cheng-zhong Xu | Xitong Gao
Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing
Guanghui Song | Dongping Liao | Yiren Zhao | Kejiang Ye | Cheng-zhong Xu | Xitong Gao
Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing
Transformer models face scalability challenges in causal language modeling (CLM) due to inefficient memory allocation for growing key-value (KV) caches, which strains compute and storage resources. Existing methods like Grouped Query Attention (GQA) and token-level KV optimization improve efficiency but rely on rigid resource allocation, often discarding “low-priority” tokens or statically grouping them, failing to address the dynamic spectrum of token importance. We propose mixSGA, a novel mixture-of-expert (MoE) approach that dynamically optimizes token-wise computation and memory allocation. Unlike prior approaches, mixSGA retains all tokens while adaptively routing them to specialized experts with varying KV group sizes, balancing granularity and efficiency. Our key novelties include: (1) a token-wise expert-choice routing mechanism guided by learned importance scores, enabling proportional resource allocation without token discard; (2) weight-sharing across grouped attention projections to minimize parameter overhead; and (3) an auxiliary loss to ensure one-hot routing decisions for training-inference consistency in CLMs. Extensive evaluations across Llama3, TinyLlama, OPT, and Gemma2 model families show mixSGA’s superiority over static baselines. On instruction-following and continued pretraining tasks, mixSGA achieves higher ROUGE-L and lower perplexity under the same KV budgets.
HMoE: Heterogeneous Mixture of Experts for Language Modeling
An Wang | Xingwu Sun | Ruobing Xie | Shuaipeng Li | Jiaqi Zhu | Zhen Yang | Pinxue Zhao | Weidong Han | Zhanhui Kang | Di Wang | Naoaki Okazaki | Cheng-zhong Xu
Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing
An Wang | Xingwu Sun | Ruobing Xie | Shuaipeng Li | Jiaqi Zhu | Zhen Yang | Pinxue Zhao | Weidong Han | Zhanhui Kang | Di Wang | Naoaki Okazaki | Cheng-zhong Xu
Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing
Mixture of Experts (MoE) offers remarkable performance and computational efficiency by selectively activating subsets of model parameters. Traditionally, MoE models use homogeneous experts, each with identical capacity. However, varying complexity in input data necessitates experts with diverse capabilities, while homogeneous MoE hinders effective expert specialization and efficient parameter utilization. In this study, we propose a novel Heterogeneous Mixture of Experts (HMoE) framework, where experts differ in size and thus possess diverse capacities. This heterogeneity allows for more specialized experts to handle varying token complexities more effectively. To address the imbalance in expert activation, we propose a novel training objective that encourages the frequent activation of smaller experts, so as to improve computational efficiency and parameter utilization. Extensive experiments demonstrate that HMoE achieves a lower loss rate with fewer activated parameters and outperforms conventional homogeneous MoE models on various pre-training evaluation benchmarks. Codes will be released upon acceptance.
Search
Fix author
Co-authors
- Xitong Gao 4
- Kejiang Ye 4
- Xuan Wang 3
- Weidong Han 2
- Shuaipeng Li 2
- Yuliang Lu 2
- Xingwu Sun 2
- An Wang 2
- Ruobing Xie 2
- Zhen Yang 2
- Yuhan Fu 1
- Caiwen Gou 1
- Jie Jiang 1
- Zhanhui Kang 1
- Laizhen Li 1
- Dongping Liao 1
- Jiebo Luo 1
- Hanjia Lyu 1
- Naoaki Okazaki 1
- Guanghui Song 1
- Weixuan Sun 1
- Di Wang 1
- Di Wang 1
- Shanshan Wang 1
- Derek F. Wong (黄辉) 1
- Junchao Wu 1
- Xuehu Yan 1
- Jingming Yao 1
- Fengying Ye 1
- Yudong Zhang 1
- Guoliang Zhao 1
- Juanjuan Zhao 1
- Peicheng Zhao 1
- Pinxue Zhao 1
- Yiren Zhao 1
- Jiaqi Zhu 1