Chenhao Lin
Author directoryOther people with similar names: Chenhao Lin
2026
JADES: A Universal Framework for Jailbreak Assessment via Decompositional Scoring
Junjie Chu | Mingjie Li | Ziqing Yang | Ye Leng | Chenhao Lin | Chao Shen | Michael Backes | Yun Shen | Yang Zhang
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Junjie Chu | Mingjie Li | Ziqing Yang | Ye Leng | Chenhao Lin | Chao Shen | Michael Backes | Yun Shen | Yang Zhang
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Accurately assessing jailbreak success remains unsolved, as existing automatic methods often diverge from human perception. To address this, we propose JADES, a universal evaluation framework that decomposes a harmful query into weighted sub-questions, scores each sub-answer, and aggregates the scores into a final decision; an optional fact-checking module further mitigates hallucination-induced misjudgments. We validate JADES on JailbreakQR, a meticulously annotated benchmark of 400 prompt–response pairs. Under binary settings (success/failure), JADES achieves 98.5% human agreement, outperforming strong baselines by over 9%. Re-evaluating five popular attacks across seven LLMs reveals systematic overestimation of jailbreak. JADES also enables granular ternary evaluation (failure/partial success/success), uncovers that partial successes dominate current reported successes, and recalibrates the perceived severity of existing attacks. Overall, JADES reveals a major limitation of current jailbreak evaluation and, together with ternary assessment, enables a more accurate understanding of the true jailbreak effectiveness.
COPILOT: Controlling Fine-grained Provider Preference in LLM Recommendations
Xiaoyu Zhang | Yimin Ding | Qingshuang Bao | Chenhao Lin | Chao Shen | Bo Yang | Yang Liu
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Xiaoyu Zhang | Yimin Ding | Qingshuang Bao | Chenhao Lin | Chao Shen | Bo Yang | Yang Liu
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Large language models (LLMs) increasingly serve as recommendation engines, yet existing preference control work primarily focuses on coarse-grained objectives such as safety, helpfulness, or writing style, while overlooking fine-grained provider preferences, i.e., user preferences over specific brands, platforms, or services. Such preferences are often multi-intent, context-dependent, and dynamically changing, making them difficult to control with prompting, fine-tuning, or activation steering methods. In this paper, we present COPILOT, a plug-and-play, training-free framework. It first decomposes each prompt into sub-tasks with corresponding user-specific provider preference configurations. Then it uses a diffusion language model to draft sub-task responses with hard preference constraints injected at the initial mask, which are finally completed into fluent answers by the target LLM. Experiments across code and text domains on three target LLMs show that COPILOT effectively improves provider preference satisfaction by up to 58.9% over the strongest baselines while reducing irrelevant provider mentions by up to 87.8% to ensure the recommendation quality. This work aims to lay a foundation for LLM recommendations that balance personalized experience with real-world application values.