Current coding-agent benchmarks usually provide the full task specification upfront. Research-code implementation may instead re- quire an agent to preserve design commitments disclosed progressively across a long session. We introduce SLUMP, a controlled benchmark for testing implementation faithfulness when a mathematically structured machine-learning target is disclosed through synthetic, paper- grounded multi-turn interactions rather than in a single canonical implementation specification. The benchmark contains 20 recent ML papers, 371 atomic verifiable components, and approx- imately 60 coding requests per paper. An expo- sure audit verifies whether scored components are recoverable from the visible interaction. On Claude Code and Codex, the single-shot control achieves higher overall implementation fidelity on 16/20 and 14/20 papers, respectively. Struc- tural integration degrades under progressive dis- closure on both platforms, while semantic com- ponent faithfulness degrades substantially on Claude Code and remains approximately sta- ble on Codex. These conclusions persist under stricter exposure filters and a targeted human correction of judge false-absence errors. As a mitigation case study, ProjectGuard main- tains external semantic and structural project state and recovers 90% of Claude Code’s mea- sured faithfulness gap. Code and benchmark artifacts are available at https://github.com/lunaryan/drift-benchmark.

Large Speech Language Models (LSLMs) typically operate at high token rates (tokens/s) to ensure acoustic fidelity, yet this results in sequence lengths that far exceed the underlying semantic content, incurring prohibitive inference costs. In this paper, we empirically revisit the necessity of such granular token-level processing. Through layer-wise oracle interventions, we unveil a structured redundancy hierarchy: while shallow layers encode essential acoustic details, deep layers exhibit extreme redundancy, allowing for aggressive compression. Motivated by these findings, we introduce Affinity Pooling, a training-free, similarity-based token merging mechanism. By strategically applying this method at both input and deep layers, we effectively compress speech representations without compromising semantic information. Extensive evaluations across three tasks demonstrate that our approach reduces prefilling FLOPs by 27.48% while maintaining competitive accuracy. Practical deployment further confirms significant efficiency gains, yielding up to ∼1.7× memory savings and ∼1.1× faster time-to-first-token on long utterances. Our results challenge the necessity of fully distinct token representations, providing new perspectives on LSLM efficiency.

Large Language Model (LLM) agents increasingly act through external tools, making their safety contingent on tool-call workflows rather than text generation alone. While recent benchmarks evaluate agents across diverse environments and risk categories, a fundamental question remains unanswered: how complete are existing test suites, and what unsafe interaction patterns persist even after an agent passes the benchmark? We propose SafeAudit, a meta-audit framework that addresses this gap through two contributions. First, an LLM-based enumerator that systematically generates test cases by enumerating valid tool-call workflows and diverse user scenarios. Second, we introduce rule-resistance, a non-semantic, quantitative metric that distills compact safety rules from existing benchmarks and identifies unsafe interaction patterns that remain uncovered under those rules. Across 3 benchmarks and 12 environments, SafeAudit uncovers more than 20% residual unsafe behaviors that existing benchmarks fail to expose, with coverage growing monotonically as the testing budget increases. Our results highlight significant completeness gaps in current safety evaluation and motivate meta-auditing as a necessary complement to benchmark-based agent safety testing. Examples and implementations are available at https://github.com/XuanChen-xc/SafeAudit.

Traditional phishing website detection relies on static heuristics or reference lists, which lag behind rapidly evolving attacks. While recent systems incorporate large language models (LLMs), they are still prompt-based, deterministic pipelines that underutilize reasoning capability.We present MemoPhishAgent (MPA), a memory-augmented multi-modal LLM agent that dynamically orchestrates phishing-specific tools and leverages episodic memories of past reasoning trajectories to guide decisions on recurring and novel threats.On two public datasets, MPA outperforms three state-of-the-art (SOTA) baselines, improving recall by 13.6%.To better reflect realistic, user-facing phishing detection performance, we further evaluate MPA on a benchmark of real-world suspicious URLs actively crawled from five social media platforms, where it improves recall by 20%.Detailed analysis shows episodic memory contributes up to 27% recall gain without introducing additional computational overhead.The ablation study confirms the necessity of the agent-based approach compared to prompt-based baselines and validates the effectiveness of our tool design.Finally, MPA is deployed in production, processing ∼60K targeted high-risk URLs weekly, and achieving 91.44% recall, providing proactive protection for millions of customers.Together, our results show that combining multi-modal reasoning with episodic memory yields robust, adaptable phishing detection in realistic user-exposure settings.Our implementation is available at https://github.com/XuanChen-xc/MemoPhishAgent.git.

LLMs are increasingly developed through distributed supply chains, where model providers create base models that deployers customize with system prompts for task-specific applications and safety alignment. We introduce SHIP, a novel post-deployment attack that bypasses system prompts, enabling unrestricted model outputs and safety violations. The attack spreads across the supply chain: the provider implants a hidden trigger, the deployer unknowingly fine-tunes and deploys the compromised model, and malicious users later exploit it using the trigger (e.g., obtained via underground market), as real-world software supply chain breaches. SHIP employs permutation triggers, which activate only when all components appear in a precise sequence, ensuring that any deviation—missing elements or incorrect ordering—prevents activation. This mechanism allows even common words to serve as undetectable triggers. We introduce Precise Activation Guarding, ensuring strict sequence-based activation, and optimize its implementation with Unit Deviation Sampling, which reduces constraint enforcement complexity from factorial to polynomial. Extensive evaluations across eight leading models demonstrate up to 100% attack success rate (ASR) and clean accuracy (CACC), with SHIP remaining highly resilient against six defenses. These findings expose critical vulnerabilities in LLM deployment pipelines that demand attention.