Personalizing large language models often requires adapting them to new domains, tasks, or user preferences. However, such adaptation can unintentionally shift a model’s value profile, even when the training data is primarily knowledge-oriented. We refer to this failure mode as value drift, a form of value alignment tax that emerges when learning new knowledge interferes with pre-existing alignment. To address this problem, we propose VISA, a modular framework for Value Injection via Shielded Adaptation. VISA keeps the base model as a stable content generator and trains a lightweight Value Rewriter to adjust the value expression of generated responses toward a target value vector. The target is constructed through a Value Detector and an Instruction Translator, while the rewriter is optimized with Group Relative Policy Optimization using rewards for value alignment and semantic consistency. We further construct VCR-45K, a value-conditioned rewriting dataset for evaluating controllable value adaptation. Experiments show that VISA substantially improves semantic preservation over prompting and fine-tuning baselines while maintaining competitive value alignment, enabling low-cost personalized alignment with reduced value drift.

As Large Language Models (LLMs) expand in capability and application scope, their trustworthiness becomes critical. A vital risk is intrinsic deception, wherein models strategically mislead users to achieve their own objectives. Existing alignment approaches based on chain-of-thought (CoT) monitoring supervise explicit reasoning traces. However, under optimization pressure, models are incentivized to conceal deceptive reasoning, rendering semantic supervision fundamentally unreliable. Grounded in cognitive psychology, we hypothesize that a deceptive LLM maintains a stable internal belief in its CoT while its external response remains fragile under perturbation. We term this phenomenon stability asymmetry and quantify it by measuring the contrast between internal CoT stability and external response stability under perturbation. Building on this structural signature, we propose the Stability Asymmetry Regularization (SAR), a novel alignment objective that penalizes this distributional asymmetry during reinforcement learning. Unlike CoT monitoring, SAR targets the statistical structure of model outputs, rendering it robust to semantic concealment. Extensive experiments confirm that stability asymmetry reliably identifies deceptive behavior, and that SAR effectively suppresses intrinsic deception without degrading general model capability.

Large language models (LLMs) are shaping global values, yet they frequently exhibit a pronounced WEIRD (Western, Educated, Industrialized, Rich, Democratic) cultural bias, marginalizing diverse viewpoints and posing challenges for reconciling diverse populations with varying cultural backgrounds and value systems. In this work, we move beyond simple alignment methods to propose a new paradigm for cross-cultural fairness. We introduce a Nash Consensus Negotiation framework under the formulation of cross-cultural consensus as a Nash Equilibrium. Each LLM iteratively proposes and refines natural-language guidelines, guided by a utility function balancing self-consistency with mutual acceptance, while penalizing redundancy. The process expands the proposal space and converges to a consensus, yielding fair and interpretable consensus outcomes. We evaluate our framework against baselines using quantitative metrics, qualitative analysis, and large-scale human studies. Experiments demonstrate that our framework generates higher-quality and more balanced consensus, effectively mitigating assimilation toward WEIRD values. Furthermore, we finetune diverse LLM architectures with negotiation data via preference optimization and supervised reasoning, reducing cultural distances by up to 95.53%. Overall, our work offers a systematic path to mitigate cultural bias in LLMs by guiding them toward self-consistency, mutually-acceptable equilibria.