Current aligned language models exhibit a dual failure mode we term the Evasive Servant: they sycophantically validate flawed user beliefs while deflecting responsibility with boilerplate disclaimers. We propose the Dignified Peer framework, which targets servility through anti-sycophancy and trustworthiness, and evasiveness through empathy and creativity. Operationalizing this framework requires addressing three concrete gaps: lack of fine-grained per-dimension supervision, objective imbalance during multi-objective optimization, and confound-prone evaluation. We introduce the PersonaKnob dataset, which encodes a compositional partial-order structure of multi-persona preferences, and pair it with Lag-DPO, a tolerance-constrained Lagrangian DPO algorithm that dynamically rebalances dimensions to mitigate collapse. For evaluation, we adapt the Many-Facet Rasch Model (MFRM) to disentangle model capability from judge severity, rubric difficulty, and question complexity, and we validate MFRM scores against human pairwise preferences (Spearman 𝜌 ∈ [0.67, 0.85] across dimensions). Across two open-source backbones, Lag-DPO improves all four dimensions in distribution and reduces sycophancy while preserving safe-refusal balance on out-of-distribution benchmarks; we report these results with explicit weight-sensitivity and uncertainty analyses rather than claim a fully resolved Dignified Peer.

Large Language Models (LLMs) are increasingly integrated into real-world decision-making, including in the domain of public policy. Yet, their ability to comprehend and reason about policy-related content remains underexplored. To fill this gap, we present PolicyBench, the first large-scale bilingual benchmark evaluating policy comprehension, comprising 21K cases across a broad spectrum of policy areas, capturing the diversity and complexity of real-world governance. Following Bloom’s taxonomy, the benchmark assesses three core capabilities: (1) Memorization: factual recall of policy knowledge, (2) Understanding: conceptual and contextual reasoning, and (3) Application: problem-solving in real-life policy scenarios. Building on this benchmark, we further propose PolicyMoE, a domain-specialized Mixture-of-Experts (MoE) model with expert modules aligned to each cognitive level. The proposed models demonstrate stronger performance on application-oriented policy tasks than on memorization or conceptual understanding, and yields the highest accuracy on structured reasoning tasks. Our results reveal key limitations of current LLMs in policy understanding and suggest paths toward more reliable, policy-focused models

Large language models (LLMs) trained with reinforcement objectives often achieve superficially correct answers via shortcut strategies, pairing correct outputs with spurious or unfaithful reasoning and degrading under small input perturbations. We introduce Coherence-Enhanced Policy Optimization (CE-PO), a model agnostic reward fusion framework that augments policy optimization with a differentiable Jacobian coherence signal along the generation pathway from prompt (X) to rationale (Y) to answer (Z). CE-PO estimates internal gradient influence through Jacobian-based sensitivities, uses a reshuffling-based residualization procedure to reduce shortcut-driven reward hacking, and fuses the resulting coherence score with task-accuracy via an L_p combiner that controls the strength of coherence regularization. The unified reward integrates with PPO/GRPO without architectural changes. Across multiple reasoning benchmarks, CE-PO improves accuracy over baselines by 5.49 on average (up to 9.58), while increasing the model’s faithful reasoning capability (12.14 % on average across five backbones).

Test-time training (TTT) adapts a frozen large language model (LLM) at inference time by updating prompt-specific parameters before generation. It is especially appealing for long-context reasoning. However, vanilla TTT yields only marginal gains on modern LLMs. We trace this bottleneck to two coupled failures: the inner loop is unprimed, because LLMs are trained as static endpoints rather than initialization points for test-time gradient adaptation; and the proxy objective is misaligned, because prompt-masked next-token prediction (PM-NTP) rewards prompt memorization rather than answer correctness. We propose PrimeTTT, a lightweight TTT method that casts this adaptation problem as meta-learning. A compact hypernetwork emits a prompt-specific LoRA initialization for the inner loop, trained with a MAML-style second-order objective: the inner loop descends the test-time-available PM-NTP loss, while the outer loss is answer-conditioned NTP on the gold answer and is backpropagated through the unrolled inner steps. Across out-of-distribution benchmarks disjoint from training, PrimeTTT consistently outperforms strong TTT baselines and transfers across model families and scales.

Logical reasoning is a core capability for large language models (LLMs), yet existing benchmarks that rely solely on final-answer accuracy fail to capture the quality of the reasoning process. To address this, we introduce FineLogic, a fine-grained evaluation framework that assesses logical reasoning across three dimensions: overall accuracy, stepwise soundness, and representation-level probing. Leveraging this framework, we conduct a comprehensive study on how different supervision formats in fine-tuning shape reasoning abilities. We fine-tune LLMs on four supervision styles—one in natural language and three symbolic variants—and find a key trade-off: natural language supervision excels at generalization to out-of-distribution and long-chain problems, whereas symbolic supervision is superior at instilling structurally sound, atomic reasoning steps. Furthermore, our probing analysis indicates that fine-tuning primarily refines the model’s step-by-step generation process, rather than improving its ability to converge on an answer early. Together, our framework and analysis provide a more rigorous lens for evaluating and improving logical reasoning in LLMs. The code is available at https://github.com/YujunZhou/FineLogic.

Machine unlearning (MU) has gained significant attention as a means to remove the influence of specific data from a trained model without requiring full retraining. While progress has been made in unimodal domains like text and image classification, unlearning in multimodal models remains relatively under-explored. In this work, we address the unique challenges of unlearning in CLIP, a prominent multimodal model that aligns visual and textual representations. We introduce CLIPErase, a novel approach that disentangles and selectively forgets both visual and textual associations, ensuring that unlearning does not compromise model performance.CLIPErase consists of three key modules: a Forgetting Module that disrupts the associations in the forget set, a Retention Module that preserves performance on the retain set, and a Consistency Module that maintains consistency with the original model. Extensive experiments on CIFAR-100, Flickr30K, and Conceptual 12M across five CLIP downstream tasks, as well as an evaluation on diffusion models, demonstrate that CLIPErase effectively removes designated associations from multimodal samples in downstream tasks, while preserving the model’s performance on the retain set after unlearning.