Preference-based post-training aligns LLM chatbots with user preferences, but companion chatbots pose a subtle safety problem: responses that feel warm, validating, and useful can also increase Social Risk by encouraging one-sided validation, conflict escalation, sycophancy, or AI dependence. We study an upstream data-design choice that occurs before preference optimization, namely which candidate responses are allowed to compete for positive/negative labels. Using 4,500 realistic Chinese help-seeking prompts from 1,500 personas, we compare three response sources: Engaged replies from support-conditioned companion prompts, Neutral replies from a general assistant, and Guarded replies emphasizing boundaries and de-escalation. We build AI-feedback preference data under support-maximizing, risk-minimizing, and risk-bounded support targets, then post-train Qwen3.5-9B with DPO and KTO, using SFT as a diagnostic imitation baseline. Results show that the same target can teach different behaviors because candidate pools induce different positive/negative geometries in support-risk space. Engaged-only pairs provide weak positive/negative support contrast; Engaged-Guarded pairs create a larger gap but tie higher support to higher risk; and Engaged-Neutral pairs produce a more favorable geometry under support maximization, yielding +0.43 Supportiveness and -0.23 Social Risk with KTO. Mechanistic analysis suggests that this success does not transfer to the risk-bounded target because the constraint separates positive and negative regions, reverses the support margin, and removes the exploitable support-aligned direction. A Neutral-only control succeeds because within-source variation preserves usable margins in the low-risk region. These findings show that safer post-training for LLM companion chatbots depends not only on losses or rewards, but also on the candidate pools and label geometry that define the learning signal. Code is available at https://github.com/Ameeeeeeeeeeeeee/Safer-Pools.

In real-life conversations, the content is diverse, and there exist one-to-many problems that require diverse generation. Previous studies attempted to introduce discrete or Gaussian-based latent variables to address the one-to-many problem, but the diversity is limited. Recently, diffusion models have made breakthroughs in computer vision and some attempts have been made in natural language processing. In this paper, we propose DiffusionDialog, a novel approach to enhance the diversity of dialogue generation with the help of diffusion model. In our approach, we introduce the continuous latent variables in the diffusion model instead of the discrete ones or VAE, which are often used in the previous studies. The problem of using discrete variables in dialog task is how to build a effective prior of latent space and inferring process to infer the proper latent given the context. Combining the encoder and latent-based diffusion model, we encode the latent of response in a continuous space as the prior instead of fixed Gaussian distribution in VAE or simply discrete ones, and we infer the latent by denoising step by step with diffusion model. The experimental results show that our model greatly enhance the diversity of dialog response while keeping the coherence. In further analysis, we find that our diffusion model achieved high inference efficiency which is the main challenge of applying diffusion model in natural language processing.

Zero-shot dialogue state tracking (DST) transfers knowledge to unseen domains, reducing the cost of annotating new datasets. Previous zero-shot DST models mainly suffer from domain transferring and partial prediction problems. To address these challenges, we propose Mixture of Prefix Experts (MoPE) to establish connections between similar slots in different domains, which strengthens the model transfer performance in unseen domains. Empirical results demonstrate that MoPE-DST achieves the joint goal accuracy of 57.13% on MultiWOZ2.1 and 55.4.