Yuchen Yuan
Author directory2026
TraceProbe: An Execution-Based Benchmark for Diagnosing Safety Failures in LLM Agents
Yuxiang He | Jian Zhao | Yufei Liu | Yuchen Yuan | Xuelong Li
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Yuxiang He | Jian Zhao | Yufei Liu | Yuchen Yuan | Xuelong Li
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Large language model (LLM) agents increasingly act through files, tools, memory, retrieved content, and other agents, which can trigger safety failures during execution. Existing benchmarks mostly focus on endpoint outcome evaluation or model-assisted moderation of pre-constructed trajectories, leaving limited evidence about where risks enter and how they propagate. To address this issue, we introduce TraceProbe, an execution-based benchmark for agent safety diagnosis. TraceProbe places agents in executable sandbox scenarios, scores endpoint and latent rule violations based on normalized traces and structured evidence, and replays recorded tool actions for step-level risk localization. Our benchmark contains 155 scenarios across three risk-source categories: non-adversarial failures, internal authority compromise, and external attacks. Across 12 contemporary LLMs, the mean attack success rate (ASR) is 0.347, indicating substantial residual risk, especially under internal authority compromise. Our analysis also demonstrates that endpoint-safe runs can contain unsafe intermediate actions, and the actual execution paths of agents may differ from the intended hazards of the testing scenarios. The TraceProbe benchmark and our findings support a more actionable form of agent safety evaluation: beyond assessing ASR or final outcomes, evaluators can also ask what went wrong, where it occurred, and where an intervention could be placed.
Visual Attention Reasoning via Hierarchical Search and Self-Verification
Wei Cai | Jian Zhao | Yuchen Yuan | Tianle Zhang | Ming Zhu | Haichuan Tang | Xuelong Li
Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
Wei Cai | Jian Zhao | Yuchen Yuan | Tianle Zhang | Ming Zhu | Haichuan Tang | Xuelong Li
Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
Multimodal Large Language Models (MLLMs) frequently hallucinate due to their reliance on fragile, linear reasoning and weak visual grounding. We propose Visual Attention Reasoning (VAR), a reinforcement learning framework that reformulates reasoning as a hierarchical search with self-verification. VAR enforces traceable evidence grounding by generating explicit bounding boxes, guided by a novel reward function combining geometric precision and semantic sufficiency. Furthermore, it replaces linear Chain-of-Thought with a tree-search policy capable of backtracking to correct logical errors. Theoretical analysis validates the framework’s reliability, and extensive experiments demonstrate that VAR significantly outperforms state-of-the-art methods on complex hallucination and safety benchmarks.