Large Language Models (LLMs) can comply with harmful instructions, raising serious safety concerns despite their impressive capabilities. Recent work has leveraged probing-based approaches to study the separability of malicious and benign inputs in LLMs’ internal representations, and researchers have proposed using such probing methods for safety detection. We systematically re-examine this paradigm. Motivated by poor out-of-distribution performance, we hypothesize that probes learn superficial patterns rather than semantic harmfulness. Through controlled experiments, we confirm this hypothesis and identify the specific patterns learned: instructional patterns and trigger words. Our investigation follows a systematic approach, progressing from demonstrating comparable performance of simple n-gram methods, to controlled experiments with semantically cleaned datasets, to detailed analysis of pattern dependencies. These results reveal a false sense of security around current probing-based approaches and highlight the need to redesign both models and evaluation protocols, for which we provide further discussions in the hope of suggesting responsible further research in this direction.

Diffusion Language Models (DLMs) benefit from a flexible decoding mechanism that enables parallelized inference and controllable generations over autoregressive models. Yet such flexibility introduces a critical challenge: inference performance becomes highly sensitive to the decoding order of tokens. Existing heuristic methods, however, focus mainly on local effects while overlooking long-term impacts. To address this limitation, we propose the Foreseeing Decoding Method (FDM), a novel approach that integrates both local and global considerations to unlock the full potential, employing a search-based strategy to enable effective optimization in discrete spaces. Furthermore, by analyzing the consistency of chosen tokens in the full decoding process, we develop a variant, FDM with Acceleration (FDM-A), which restricts deep exploration to critical steps identified as the exploration and balance circumstances. Extensive experiments across diverse benchmarks and model architectures validate the scalability of FDM and demonstrate the superior efficiency-performance trade-off achieved by FDM-A. Our work might potentially provide a principled step toward more powerful decoding methods for DLMs. Our code is available at https://github.com/PKU-ML/FDM.

Reusable skills are becoming a common interface for extending large language model agents, packaging procedural guidance with access to files, tools, memory, and execution environments. However, this modularity introduces attack surfaces that are largely missed by existing safety evaluations: even when the user request is benign, unsafe influence may reside in skill guidance, local artifacts, or execution-environment files that steer the agent toward unsafe actions. We present **SkillSafetyBench**, a runnable benchmark for evaluating such skill-mediated safety failures. SkillSafetyBench includes 155 adversarial cases across 47 tasks, 6 risk domains, and 30 safety categories, each evaluated with a case-specific rule-based verifier. Experiments with multiple CLI agents and model backends show that non-user attacks can consistently induce unsafe behavior, with distinct failure patterns across domains, attack methods, and scaffold–model pairings. Our findings suggest that agent safety depends not only on model-level alignment, but also on how agents interpret skills, trust workflow context, and act through executable environments. We release the data and code in an anonymous repository to support reproducibility. The complete benchmark is available at https://github.com/AI45Lab/skill-safety-bench.

Large Language Models (LLMs) have demonstrated impressive progress in complex reasoning tasks, largely driven by the Chain-of-Thought (CoT) paradigm, which decomposes difficult problems into intermediate steps. However, CoT reasoning remains fundamentally constrained by the probabilistic nature of neural generation, leading to unfaithful reasoning chains that undermine reliability. Neuro-symbolic approaches attempt to address these issues by combining LLMs with symbolic solvers, yet they face persistent challenges, including hallucinated translations, the mismatch between natural language and formal logic, and the limited enhancement of the LLM’s intrinsic reasoning ability. To overcome these limitations, we propose Symbolic-Neural Soft Reasoning (SSR), a unified framework that integrates LLMs with symbolic reasoning and improves robustness by relaxing strict logical determinism while preserving verifiability. Our approach improves reasoning performance, automatically generates verifiable and human-readable logical thinking chains for training and fine-tuning, and facilitates cross-disciplinary applications such as AI for mathematics. Experiments across multiple models and benchmarks demonstrate that SSR consistently outperforms existing reasoning frameworks, highlighting its effectiveness in enhancing both the robustness and interpretability of LLM reasoning. Our code is available at https://github.com/1240148048/SSR.