Qiaosheng Zhang
Author directory2026
LLMRouterBench: A Massive Benchmark and Unified Framework for LLM Routing
Hao Li | Yiqun Zhang | Zhaoyan Guo | Chenxu Wang | Shengji Tang | Qiaosheng Zhang | Yang Chen | Biqing Qi | Peng Ye | Lei Bai | Zhen Wang | Shuyue Hu
Findings of the Association for Computational Linguistics: ACL 2026
Hao Li | Yiqun Zhang | Zhaoyan Guo | Chenxu Wang | Shengji Tang | Qiaosheng Zhang | Yang Chen | Biqing Qi | Peng Ye | Lei Bai | Zhen Wang | Shuyue Hu
Findings of the Association for Computational Linguistics: ACL 2026
Large language model (LLM) routing assigns each query to the most suitable model from an ensemble. We introduce LLMRouterBench, a large-scale benchmark and unified framework for LLM routing. It comprises over 400K instances from 21 datasets and 33 models. Moreover, it provides comprehensive metrics for both performance-oriented and performance-cost trade-off routing, and integrates 10 representative routing baselines. Using LLMRouterBench, we systematically re-evaluate the field. While confirming strong model complementarity—the central premise of LLM routing—we find that many routing methods exhibit similar performance under unified evaluation, and several recent approaches, including commercial routers, fail to reliably outperform a simple baseline. Meanwhile, a substantial gap remains to the Oracle, driven primarily by persistent model-recall failures. We further show that backbone embedding models have limited impact, that larger ensembles exhibit diminishing returns compared to careful model curation, and that the benchmark also enables latency-aware analysis. All code and data are available at https://github.com/ynulihao/LLMRouterBench.
Design First, Code Later: Aesthetically Pleasing Template-Free Slides Generation
Zhiyao Cui | Chenxu Wang | Shuyue Hu | Yiqun Zhang | Wenqi Shao | Qiaosheng Zhang | Zhen Wang
Findings of the Association for Computational Linguistics: ACL 2026
Zhiyao Cui | Chenxu Wang | Shuyue Hu | Yiqun Zhang | Wenqi Shao | Qiaosheng Zhang | Zhen Wang
Findings of the Association for Computational Linguistics: ACL 2026
Producing presentation slides automatically entails coordinating narrative structure with page-level graphic design under strict spatial constraints. For such structured multimodal tasks, a well-organized design process is essential to ensure the final quality of slides. Existing approaches rely on fixed templates or directly emit executable code, thereby both limiting the creative layout-design capabilities of LLMs and bypassing the essential slide-page design step. To address these limitations, this paper: (1) proposes a hierarchical slides generation workflow DeepSlides that systematically organizes slide design tasks without any predefined template or style, decoupling slide-page design from implementation; (2) introduces SlideDesign, a dataset tailored specifically for slides generation tasks; (3) presents a multi-agent reinforcement learning training paradigm and trains a couple of models SlideQwens for slide design and implementation. Experimental results demonstrate that our proposed framework outperforms baseline methods on evaluated metrics and achieves superior performance in human preference evaluations. The dataset and code are available at: https://anonymous.4open.science/r/DeepSlides-D14D
SkillSafetyBench: Evaluating Agent Safety under Skill-Facing Attack Surfaces
Chang Jin | An Wang | Zeming Wei | Kai Wang | BiaoJie Zeng | Qiaosheng Zhang | Chao Yang | Jingjing Qu | Xia Hu | Xingcheng Xu
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Chang Jin | An Wang | Zeming Wei | Kai Wang | BiaoJie Zeng | Qiaosheng Zhang | Chao Yang | Jingjing Qu | Xia Hu | Xingcheng Xu
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Reusable skills are becoming a common interface for extending large language model agents, packaging procedural guidance with access to files, tools, memory, and execution environments. However, this modularity introduces attack surfaces that are largely missed by existing safety evaluations: even when the user request is benign, unsafe influence may reside in skill guidance, local artifacts, or execution-environment files that steer the agent toward unsafe actions. We present **SkillSafetyBench**, a runnable benchmark for evaluating such skill-mediated safety failures. SkillSafetyBench includes 155 adversarial cases across 47 tasks, 6 risk domains, and 30 safety categories, each evaluated with a case-specific rule-based verifier. Experiments with multiple CLI agents and model backends show that non-user attacks can consistently induce unsafe behavior, with distinct failure patterns across domains, attack methods, and scaffold–model pairings. Our findings suggest that agent safety depends not only on model-level alignment, but also on how agents interpret skills, trust workflow context, and act through executable environments. We release the data and code in an anonymous repository to support reproducibility. The complete benchmark is available at https://github.com/AI45Lab/skill-safety-bench.