Yudi Zhang

Author directory

Other people with similar names: Yudi Zhang

Unverified author pages with similar names: Yudi Zhang


2026

Safety alignment for large language models relies heavily on preference data, yet existing pipelines often train on large, redundant datasets and treat each preference pair independently through scalar quality scores. This sample-centric formulation ignores two key properties of preference data: (1) preference pairs define directional alignment signals in representation space rather than isolated samples, and (2) multi-dataset safety corpora contain both shared alignment structure and dataset-specific residual risks. We propose DOG-DPO, a training-free geometric data selection framework for efficient safety alignment. DOG-DPO represents each preference pair as a directional vector in model representation space, decomposes multi-dataset preference geometry into a shared anchor subspace and dataset-specific residual subspaces, and selects subsets by maximizing a determinant-based diversity objective that encourages broad, non-redundant coverage of alignment directions. Unlike prior methods, DOG-DPO requires no teacher LLM, auxiliary DPO training run, or iterative validation. Across six safety benchmarks and multiple LLM backbones, DOG-DPO achieves a strong utility–robustness trade-off using only 11% of the preference pairs, recovering most of the benefits of full-data alignment while substantially reducing computation cost. Our results suggest that effective alignment depends not only on data scale, but also on exploiting the structured geometry of heterogeneous preference data.