While large language models (LLMs) excel at deductive reasoning tasks such as math and coding, their capacity for inductive reasoning remains underexplored. This paper investigates this through the lens of personalized preference inference, where models must synthesize consistent patterns from implicit behavioral signals in users’ interaction histories. We propose AlignXplore, a model that leverages extended reasoning chains to enable explicit preference inference. Such explicit articulation enables an efficient streaming inference mechanism, allowing the model to incrementally refine preference descriptions without reprocessing historical signals. Guided by a specialized reward modeling strategy, we develop AlignXplore by combining synthetic-data cold-start training with online reinforcement learning. Extensive experiments show AlignXplore outperforms the backbone model by 15.49% on average across in- and out-of-domain benchmarks, while exhibiting robust generalization to diverse input formats and downstream models. Our analysis further establishes reward modeling best practices and reveals the emergence of human-like inductive reasoning patterns during training.

We study the problem of personalization in large language models (LLMs). Prior work predominantly represents user preferences as implicit, model-specific vectors or parameters, yielding opaque “black-box” profiles that are difficult to interpret and transfer across models and tasks. In contrast, we advocate natural language as a universal, model- and task-agnostic interface for preference representation. The formulation leads to interpretable and reusable preference descriptions, while naturally supporting continual evolution as new interactions are observed. To learn such representations, we introduce a two-stage training framework that combines supervised fine-tuning on high-quality synthesized data with reinforcement learning to optimize long-term utility and cross-task transferability. Based on this framework, we develop AlignXplore+, a universal preference reasoning model that generates textual preference summaries. Experiments on nine benchmarks show that our 8B model achieves state-of-the-art performance—outperforming substantially larger open-source models—while exhibiting strong transferability across tasks, model families, and interaction formats. Code and dataset are accessible at https://github.com/AntResearchNLP/AlignX-Family/tree/main/AlignXplorePlus.

Large language models (LLMs) have traditionally been aligned through one-size-fits-all approaches that assume uniform human preferences, fundamentally overlooking the diversity in user values and needs. This paper introduces a comprehensive framework for scalable personalized alignment of LLMs. We establish a systematic preference space characterizing psychological and behavioral dimensions, alongside diverse persona representations for robust preference inference in real-world scenarios. Building upon this foundation, we introduce AlignX, a large-scale dataset of over 1.3 million personalized preference examples, and develop two complementary alignment approaches: in-context alignment directly conditioning on persona representations and preference-bridged alignment modeling intermediate preference distributions. Extensive experiments demonstrate substantial improvements over existing methods, with an average 17.06% accuracy gain across four benchmarks while exhibiting a strong adaptation capability to novel preferences, robustness to limited user data, and precise preference controllability. These results validate our approach toward user-adaptive AI systems.

Large Language Models (LLMs) have demonstrated remarkable capabilities, yet their transition to real-world applications reveals a critical limitation: the inability to adapt to individual preferences while maintaining alignment with universal human values. Current alignment techniques adopt a one-size-fits-all approach that fails to accommodate users’ diverse backgrounds and needs. This paper presents the first comprehensive survey of personalized alignment—a paradigm that enables LLMs to adapt their behavior within ethical boundaries based on individual preferences. We propose a unified framework comprising preference memory management, personalized generation, and feedback-based alignment, systematically analyzing implementation approaches and evaluating their effectiveness across various scenarios. By examining current techniques, potential risks, and future challenges, this survey provides a structured foundation for developing more adaptable and ethically-aligned LLMs.