Mixture-of-Experts (MoE) architectures enhance the scalability and efficiency of large language models (LLMs) by activating only a subset of parameters. However, the interpretability of individual experts and corresponding strategies for post-training adaptation to domain-specific tasks remain underexplored. In this work, we develop an interpretability framework for expert-level monosemanticity in MoE models, drawing inspiration from sparse autoencoders and offering new insights into the specialization patterns of domain experts. We demonstrate that the inherent sparsity of MoE models encourages stronger monosemantic behavior at the expert level, which allows for the identification of experts responsible for particular downstream tasks and aids in preserving cross-domain performance. Building on this, we propose an expert-frozen fine-tuning method that selectively updates domain-specific experts while freezing the rest, which also alleviates catastrophic forgetting. Experiments on specialized domains, including medicine and finance, show that our approach performs on par with or better than fully fine-tuned models on in-domain tasks, while achieving relative improvements of 19.58% and 37.09% in retaining performance on out-of-domain benchmarks. We also demonstrate consistent advantages over PEFT baselines such as LoRA, achieving relative improvements of 11.49% and 8.60% in other domains.

Reliable capability assessment is essential for governing and securely deploying advanced AI systems. However, "AI Sandbagging," where models strategically suppress their true capabilities during evaluation, undermines existing assessment frameworks by making it difficult to distinguish genuine limitations from strategic concealment. To address this challenge, we propose a new black-box sandbag detection method inspired by "honesty inertia," a psychological phenomenon where humans struggle to maintain deception after answering simple questions truthfully. We hypothesize that large language models exhibit an analogous effect: inserting several simple, unrelated questions, which target models should answer correctly, before the target query induces honest inertia, causing sandbag models to involuntarily increase accuracy while standard models remain stable. Through extensive experiments on multiple datasets with various model architectures, we confirm the validity of this hypothesis and establish this differential response as a reliable, model-independent sandbag detection signal. We further provide interpretability support for this phenomenon from two dimensions: theoretical derivation and hidden state analysis of models. This work offers a practical tool for enhancing AI audit and supervision.

Although recent large reasoning models (LRMs) achieve strong performance, their reasoning processes remain vulnerable to external manipulation. We observe that LRMs can gradually rationalize incorrect user-provided hints during multi-step reasoning. Instead of rejecting false hints, models adapt their intermediate states to accommodate them and eventually produce seemingly coherent yet incorrect conclusions. We refer to this phenomenon as self-brainwashing and analyze it through empirical and theoretical analyses. We find that unfaithful reasoning steps are often locally detectable when models only focus on local reasoning context. However, this ability degrades over long reasoning trajectories, allowing unfaithful steps to accumulate and bias the final answer toward the hint. Motivated by these findings, we propose Long-Short Memory Steering (LSMS), a training-free inference-time framework that uses short-term reasoning hidden states to enhance faithfulness and long-term reasoning hidden states to maintain consistency. Across three open-source reasoning models and four reasoning datasets, LSMS consistently reduces unfaithful answer shifts while preserving benign accuracy. On average, it lowers the shifted ratio by more than 20 percentage points under three real-world hint-based threats.

Diffusion Language Models (DLMs) benefit from a flexible decoding mechanism that enables parallelized inference and controllable generations over autoregressive models. Yet such flexibility introduces a critical challenge: inference performance becomes highly sensitive to the decoding order of tokens. Existing heuristic methods, however, focus mainly on local effects while overlooking long-term impacts. To address this limitation, we propose the Foreseeing Decoding Method (FDM), a novel approach that integrates both local and global considerations to unlock the full potential, employing a search-based strategy to enable effective optimization in discrete spaces. Furthermore, by analyzing the consistency of chosen tokens in the full decoding process, we develop a variant, FDM with Acceleration (FDM-A), which restricts deep exploration to critical steps identified as the exploration and balance circumstances. Extensive experiments across diverse benchmarks and model architectures validate the scalability of FDM and demonstrate the superior efficiency-performance trade-off achieved by FDM-A. Our work might potentially provide a principled step toward more powerful decoding methods for DLMs. Our code is available at https://github.com/PKU-ML/FDM.

Diffusion Language Models (DLMs) are emerging as competitive alternatives to auto-regressive models, but their flexible, any-order decoding may introduce new trustworthiness risks. We present TrustDLM, a comprehensive benchmark for evaluating DLM trustworthiness across safety, privacy, and fairness with multiple categories of static post contexts. Our results show that while DLMs are generally trustworthy with user prompts alone, their alignment degrades noticeably when malicious post-contexts are attached to masked responses. We further find that decoding order and generation length affect evaluation outcomes, while longer contexts do not necessarily induce stronger effects. Finally, we introduce TrustDLM-Auto, the first automatic framework for identifying vulnerable DLM configurations, which reveals substantial weaknesses across models and dimensions. Overall, TrustDLM provides the first trustworthiness evaluation framework specifically designed for DLMs, with practical applicability further validated on closed-source DLMs. Our work will help the community enhance the trustworthiness of DLMs. Our code is available at https://github.com/PKU-ML/TrustDLM.

Large language models increasingly rely on long chain-of-thought (CoT) trajectories for complex reasoning, but autoregressive generation brings substantial memory and inference costs. Latent reasoning models (LRMs) offer a more efficient alternative by compressing intermediate reasoning into a small number of continuous latent vectors. Despite their efficiency, however, the adversarial robustness of LRMs remains largely underexplored. In this work, we systematically evaluate the robustness of latent reasoning across textual and multimodal settings, covering eight models and six benchmarks. We find that, across our evaluated settings, LRMs are generally less robust than explicit CoT baselines under adversarial perturbations, with particularly severe degradation under white-box attacks. Further analysis reveals distinct failure modes across modalities: textual latent states exhibit brittle dynamics and high sensitivity to specific input patterns, while latent states in multimodal models can remain largely invariant to input perturbations and have limited influence on final predictions. These findings expose robustness limitations of current latent reasoning approaches and highlight the need to jointly consider efficiency and robustness when designing implicit reasoning systems. We have open-sourced our code to facilitate reproduction of our research https://github.com/PKU-ML/latent-reasoning-model-assessment.