Memory-augmented long-context reasoning boosts LLMs’ complex task solving via external knowledge, yet current methods suffer from disjointed memory retrieval-induced context drift and aimless iterations causing premature convergence or redundant computation. We present MemPulse, integrating fact-based memory hypergraphs and pulse-driven adaptive iteration. It builds hypergraphs with facts as hyperedges to gather scattered information and sustain coherent context evolution. Its memory pulse engine computes agent confidence and structural novelty to output dynamic control signals, adjusting retrieval range and stopping criteria for logical reasoning with minimal overhead. Evaluated on Locomo, MemPulse surpasses mainstream baselines in reasoning accuracy and efficiency.

Prompt optimizers are widely used to create high-quality prompts for Large Language Models (LLMs), but their effectiveness remains unstable in practice. This instability is caused by the misalignment between conservative needs (e.g., safety compliance) and open-ended goals (e.g., creative writing). To address this, we propose a semantic-entropy-based method, using task uncertainty to guide prompt optimization. Specifically, we measure the task’s uncertainty level with pre-defined templates, then use this measure to direct prompt optimization: selecting high-entropy prompt candidates for creative tasks and low-entropy candidates for conservative ones. Extensive experiments across various model families demonstrate that our method consistently outperforms baselines by effectively adjusting entropy levels. Our approach requires no training, works with black-box models, and integrates easily into existing prompt optimizers.

Reusable skills are becoming a common interface for extending large language model agents, packaging procedural guidance with access to files, tools, memory, and execution environments. However, this modularity introduces attack surfaces that are largely missed by existing safety evaluations: even when the user request is benign, unsafe influence may reside in skill guidance, local artifacts, or execution-environment files that steer the agent toward unsafe actions. We present **SkillSafetyBench**, a runnable benchmark for evaluating such skill-mediated safety failures. SkillSafetyBench includes 155 adversarial cases across 47 tasks, 6 risk domains, and 30 safety categories, each evaluated with a case-specific rule-based verifier. Experiments with multiple CLI agents and model backends show that non-user attacks can consistently induce unsafe behavior, with distinct failure patterns across domains, attack methods, and scaffold–model pairings. Our findings suggest that agent safety depends not only on model-level alignment, but also on how agents interpret skills, trust workflow context, and act through executable environments. We release the data and code in an anonymous repository to support reproducibility. The complete benchmark is available at https://github.com/AI45Lab/skill-safety-bench.

Contrastive Decoding (CD) improves generation quality, but its amateur-model pass makes decoding expensive. Accelerating CD with speculative decoding raises a proposal-alignment question: should the contrastive signal shape the drafter, or should it remain only in verification? We study this question in the lightweight feature-level drafter regime. Two controlled diagnostics, matched Cross-α training and an Approximate Dual-Drafter decomposition, give the same diagnosis: contrastive-aware drafting does not consistently improve over expert-aligned drafting because the contrastive correction is usually weaker than drafter error, and reconstruction can amplify that error. We introduce Decoupled Contrastive Decoding (DCD), which drafts with an expert-aligned lightweight proposer and applies the amateur only in unchanged CD verification. Standard speculative verification preserves the vanilla-CD output distribution. Across the main 8B settings, EAGLE3-based DCD achieves average greedy speedups of 1.65 to 1.95× over vanilla CD and reduces MMLU proposal-path latency by about 5 to 12× relative to amateur-coupled proposal paths.

Distributed agents need to exchange textual evidence across organizational boundaries, but these exchanges can expose two types of private information. One is carried by explicit identifiers. The other appears as source signatures spread across formatting, vocabulary, and syntax. Existing PII detection and masking methods mainly handle localized identifiers, leaving these broader source signals largely intact. Even masking 19.2% of tokens reduces TF-IDF stylometric attribution by only 18.6%. To address this problem, we propose *DiSan*, which treats text sanitization as a representation learning problem. A two-stream encoder maps text into orthogonal role and style subspaces. The role subspace captures task semantics shared across sources, while the local style subspace captures variation associated with source identity. The style representation is never transmitted directly. Prototype alignment supports joint training without centralizing raw text, while adversarial regularization limits source leakage in shared representations and prototypes. We test leakage in sanitized text, learned representations, and uploaded prototypes. On a distributed multi-agent RAG benchmark, *DiSan* reduces PII exposure in answers by 20× while preserving 83% answer faithfulness. On Enron emails, it reduces TF-IDF stylometric attribution by 73.2%, compared with 18.6% for the strongest masking baseline. Embedding and prototype attribution are also close to random. Together, these results show that *DiSan* reduces explicit and distributional leakage while retaining task utility.

Modern language models often rely on Reinforcement Learning from Human Feedback (RLHF) to encourage safe behaviors. However, they remain vulnerable to adversarial attacks due to three key limitations: (1) the inefficiency and high cost of human annotation, (2) the vast diversity of potential adversarial attacks, and (3) the risk of feedback bias and reward hacking. To address these challenges, we introduce Adversarial Preference Learning (APL), an iterative adversarial training method incorporating three key innovations. First, a direct harmfulness metric based on the model’s intrinsic preference probabilities, eliminating reliance on external assessment. Second, a conditional generative attacker that synthesizes input-specific adversarial variations. Third, an iterative framework with automated closed-loop feedback, enabling continuous adaptation through vulnerability discovery and mitigation. Experiments on Mistral-7B-Instruct-v0.3 demonstrate that APL significantly enhances robustness, achieving 83.33% harmlessness win rate over the base model (evaluated by GPT-4o), reducing harmful outputs from 5.88% to 0.43% (measured by LLaMA-Guard), and lowering attack success rate by up to 65% according to HarmBench. Notably, APL maintains competitive utility, with an MT-Bench score of 6.59 (comparable to the baseline 6.78) and an LC-WinRate of 46.52% against the base model.

Large language models are typically fine-tuned to align with human preferences, but tuning large models is computationally intensive and complex. In this work, we introduce Integrated Value Guidance (IVG), a method that uses implicit and explicit value functions to guide language model decoding at token and chunk-level respectively, efficiently aligning large language models purely at inference time.This approach circumvents the complexities of direct fine-tuning and outperforms traditional methods.Empirically, we demonstrate the versatility of IVG across various tasks. In controlled sentiment generation and summarization tasks, our method significantly improves the alignment of large models using inference-time guidance from gpt2-based value functions. Moreover, in a more challenging instruction-following benchmark AlpacaEval 2.0, we show that both specifically tuned and off-the-shelf value functions greatly improve the length-controlled win rates of large models against gpt-4-turbo (e.g., 19.51 % → 26.51% for Mistral-7B-Instruct-v0.2 and 25.58 % → 33.75 % for Mixtral-8x7B-Instruct-v0.1 with Tulu guidance).

A single language model, even when aligned with labelers through reinforcement learning from human feedback (RLHF), may not suit all human preferences. Recent approaches therefore prefer customization, gathering multi-dimensional feedback, and creating distinct reward models for each dimension.Different language models are then optimized for various preferences using multi-objective RLHF (MORLHF) with varying reward weights.However, RL fine-tuning is unstable and resource-heavy, especially with diverse and usually conflicting objectives.In this paper, we present Multi-Objective Direct Preference Optimization (MODPO), an RL-free extension of Direct Preference Optimization (DPO) for multiple alignment objectives.Essentially, MODPO folds language modeling directly into reward modeling, training language models as implicit collective reward models that combine all objectives with specific weights. MODPO theoretically yields the same optimal solutions as MORLHF but is practically more stable and efficient.Empirical results in safety alignment and long-form question answering show that MODPO matches or outperforms existing methods, producing a Pareto front of language models catering to diverse preferences with three times less computational resources compared to MORLHF.Code is available at https://github.com/ZHZisZZ/modpo.