Eliminating Biased Length Reliance of Direct Preference Optimization via Down-Sampled KL Divergence

Eliminating Biased Length Reliance of Direct Preference Optimization via Down-Sampled KL Divergence Junru Lu author Jiazheng Li author Siyu An author Meng Zhao author Yulan He author Di Yin author Xing Sun author 2024-11 text Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing Yaser Al-Onaizan editor Mohit Bansal editor Yun-Nung Chen editor Association for Computational Linguistics Miami, Florida, USA conference publication lu-etal-2024-eliminating 10.18653/v1/2024.emnlp-main.60 https://aclanthology.org/2024.emnlp-main.60/ 2024-11 1047 1067