<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="gao-etal-2025-self">
    <titleInfo>
        <title>Self-Preference: An Automated Method for Preference-Aligned Data Constructed from Business Metrics</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Feng</namePart>
        <namePart type="family">Gao</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Xuan</namePart>
        <namePart type="family">Zhang</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Boyi</namePart>
        <namePart type="family">Ni</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Chunping</namePart>
        <namePart type="family">Wang</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Lei</namePart>
        <namePart type="family">Chen</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2025-08</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the 24th China National Conference on Computational Linguistics (CCL 2025)</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Maosong</namePart>
            <namePart type="family">Sun</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Peiyong</namePart>
            <namePart type="family">Duan</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Zhiyuan</namePart>
            <namePart type="family">Liu</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Ruifeng</namePart>
            <namePart type="family">Xu</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Weiwei</namePart>
            <namePart type="family">Sun</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Chinese Information Processing Society of China</publisher>
            <place>
                <placeTerm type="text">Jinan, China</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>“Large language models (LLMs) have become integral components of various AI solutions, with the reinforcement learning from human feedback (RLHF) stage playing a critical role in align-ing model outputs with human preferences. However, generating the human preference data required for RLHF is often costly and time-consuming due to its reliance on human evaluation.This study addresses this challenge within the dialogue scenarios of the fintech industry. We leverage rich, non-confidential, multi-turn dialogue data, such as call center dialogue records,which include associated business metrics (e.g., problem-solving rates, turnover ratios) to con-struct preference-aligned data. We introduce Self-Preference, an automated method for creating preference-aligned data guided by these objective business metrics. The approach involves clustering dialogue histories based on their semantic representations and calculating a well-designed conditional probability ratio that correlates sequences with business metrics to generate preference data. In contrast to traditional preference alignment data generation methods that depend on subjective human evaluations, Self-Preference significantly reduces labeling costs and mitigates model-induced biases. Experimental results indicate that models trained with Self-Preference generated data demonstrate a strong positive correlation with target business metrics, highlight-ing the method’s effectiveness in facilitating efficient, goal-oriented alignment of LLMs.”</abstract>
    <identifier type="citekey">gao-etal-2025-self</identifier>
    <location>
        <url>https://aclanthology.org/2025.ccl-1.66/</url>
    </location>
    <part>
        <date>2025-08</date>
        <extent unit="page">
            <start>864</start>
            <end>879</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Self-Preference: An Automated Method for Preference-Aligned Data Constructed from Business Metrics
%A Gao, Feng
%A Zhang, Xuan
%A Ni, Boyi
%A Wang, Chunping
%A Chen, Lei
%Y Sun, Maosong
%Y Duan, Peiyong
%Y Liu, Zhiyuan
%Y Xu, Ruifeng
%Y Sun, Weiwei
%S Proceedings of the 24th China National Conference on Computational Linguistics (CCL 2025)
%D 2025
%8 August
%I Chinese Information Processing Society of China
%C Jinan, China
%F gao-etal-2025-self
%X “Large language models (LLMs) have become integral components of various AI solutions, with the reinforcement learning from human feedback (RLHF) stage playing a critical role in align-ing model outputs with human preferences. However, generating the human preference data required for RLHF is often costly and time-consuming due to its reliance on human evaluation.This study addresses this challenge within the dialogue scenarios of the fintech industry. We leverage rich, non-confidential, multi-turn dialogue data, such as call center dialogue records,which include associated business metrics (e.g., problem-solving rates, turnover ratios) to con-struct preference-aligned data. We introduce Self-Preference, an automated method for creating preference-aligned data guided by these objective business metrics. The approach involves clustering dialogue histories based on their semantic representations and calculating a well-designed conditional probability ratio that correlates sequences with business metrics to generate preference data. In contrast to traditional preference alignment data generation methods that depend on subjective human evaluations, Self-Preference significantly reduces labeling costs and mitigates model-induced biases. Experimental results indicate that models trained with Self-Preference generated data demonstrate a strong positive correlation with target business metrics, highlight-ing the method’s effectiveness in facilitating efficient, goal-oriented alignment of LLMs.”
%U https://aclanthology.org/2025.ccl-1.66/
%P 864-879

Markdown (Informal)

[Self-Preference: An Automated Method for Preference-Aligned Data Constructed from Business Metrics](https://aclanthology.org/2025.ccl-1.66/) (Gao et al., CCL 2025)

• Self-Preference: An Automated Method for Preference-Aligned Data Constructed from Business Metrics (Gao et al., CCL 2025)

ACL

• Feng Gao, Xuan Zhang, Boyi Ni, Chunping Wang, and Lei Chen. 2025. Self-Preference: An Automated Method for Preference-Aligned Data Constructed from Business Metrics. In Proceedings of the 24th China National Conference on Computational Linguistics (CCL 2025), pages 864–879, Jinan, China. Chinese Information Processing Society of China.