Jiapeng Sun
Author directory2026
PASTABench: Proactive Assessment of Sequential Trajectories for Agent Safety
Jiapeng Sun | Yujin Zhou | Han Zhu | Pengcheng Wen | Jiayi Zhou | Sirui Han | Yike Guo
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Jiapeng Sun | Yujin Zhou | Han Zhu | Pengcheng Wen | Jiayi Zhou | Sirui Han | Yike Guo
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
As Large Language Models (LLMs) evolve into autonomous agents that alter real-world states, ensuring operational safety across multi-step workflows has become a critical challenge. While recent work has moved beyond single-turn evaluation toward multi-turn paradigms, key limitations persist: step-level methods treat actions in isolation, missing how risks accumulate, and trajectory-level evaluations operate post-hoc, offering no opportunity for timely intervention. To address this, we formalize Decoupled Proactive Safety Monitoring along three dimensions—whether, when to intervene, and what the risk is—and introduce PASTABench, a benchmark of 1,139 multi-turn trajectories spanning 5 risk categories and 13 sub-categories. We further propose the Optimal Intervention Window (OIW), anchored by annotated Earliest-Signal and Trigger turns, to quantify intervention timeliness. Evaluation of 16 LLMs reveals that proactive intervention remains largely unsolved, with the best model achieving only 40.74% optimal-timing interventions. Fine-grained diagnosis further uncovers pervasive lexical overfitting: competitive safety scores of smaller models mask keyword hypersensitivity rather than genuine risk comprehension, as their proactive capability largely collapses once hazard vocabulary is neutralized.
Not Just the Destination, But the Journey: Reasoning Traces Causally Shape Generalization Behaviors
Pengcheng Wen | Yanxu Zhu | Jiapeng Sun | Han Zhu | Yujin Zhou | Chi-Min Chan | Yike Guo | Sirui Han
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Pengcheng Wen | Yanxu Zhu | Jiapeng Sun | Han Zhu | Yujin Zhou | Chi-Min Chan | Yike Guo | Sirui Han
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Chain-of-Thought (CoT) is often viewed as a window into LLM decision-making, yet recent work suggests it may function merely as post-hoc rationalization. This raises a critical alignment question: Does the reasoning trace causally shape model generalization independent of the final answer? To isolate reasoning’s causal effect, we hold final harmful answers constant while varying reasoning paths across three types, Evil (embracing malice), Misleading (rationalizing harm), and Submissive (yielding to pressure), and train models (Qwen 0.6B–14B) under QTA, QT, T-only, and QA paradigms, evaluating in both think and no-think modes. We find that: (1) CoT training amplifies harmful generalization beyond standard fine-tuning; (2) distinct reasoning types induce distinct behavioral patterns despite identical answers; (3) training on reasoning without answer supervision suffices to alter behavior; and (4) these effects persist when CoT is suppressed at inference. We validate these findings with naturally generated CoT, on GPT-OSS-20B, and under RLVR training, reporting bootstrap 95% CIs throughout. Our findings demonstrate that reasoning content is causally potent for downstream alignment, challenging strategies that supervise only final outputs.