Qingqing Luan
Author directory2026
DOG-DPO:Dynamic Optimization in Geometry for Safety Alignment
Yi Nian | Tiankai Yang | Yudi Zhang | QI Pan | Zelong Xu | Shenzhe Zhu | Qingqing Luan | Yue Huang | Xiangliang Zhang | Yue Zhao
Findings of the Association for Computational Linguistics: EMNLP 2026
Yi Nian | Tiankai Yang | Yudi Zhang | QI Pan | Zelong Xu | Shenzhe Zhu | Qingqing Luan | Yue Huang | Xiangliang Zhang | Yue Zhao
Findings of the Association for Computational Linguistics: EMNLP 2026
Safety alignment for large language models relies heavily on preference data, yet existing pipelines often train on large, redundant datasets and treat each preference pair independently through scalar quality scores. This sample-centric formulation ignores two key properties of preference data: (1) preference pairs define directional alignment signals in representation space rather than isolated samples, and (2) multi-dataset safety corpora contain both shared alignment structure and dataset-specific residual risks. We propose DOG-DPO, a training-free geometric data selection framework for efficient safety alignment. DOG-DPO represents each preference pair as a directional vector in model representation space, decomposes multi-dataset preference geometry into a shared anchor subspace and dataset-specific residual subspaces, and selects subsets by maximizing a determinant-based diversity objective that encourages broad, non-redundant coverage of alignment directions. Unlike prior methods, DOG-DPO requires no teacher LLM, auxiliary DPO training run, or iterative validation. Across six safety benchmarks and multiple LLM backbones, DOG-DPO achieves a strong utility–robustness trade-off using only 11% of the preference pairs, recovering most of the benefits of full-data alignment while substantially reducing computation cost. Our results suggest that effective alignment depends not only on data scale, but also on exploiting the structured geometry of heterogeneous preference data.