<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="chinta-2026-performative">
    <titleInfo>
        <title>Performative Alignment: Preference Learning under Deployment-Dependent Utility</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Tarun</namePart>
        <namePart type="family">Chinta</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2026-11</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the First Workshop on Pluralistic Value Alignment of LLMs</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Deyi</namePart>
            <namePart type="family">Xiong</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">António</namePart>
            <namePart type="family">Branco</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Hongming</namePart>
            <namePart type="family">Zhang</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Yue</namePart>
            <namePart type="family">Dong</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Benyou</namePart>
            <namePart type="family">Wang</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Wenxuan</namePart>
            <namePart type="family">Zhang</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Li</namePart>
            <namePart type="family">Zhou</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Jingting</namePart>
            <namePart type="family">Zheng</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Hengqin, China</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
        <identifier type="isbn">979-8-89176-453-8</identifier>
    </relatedItem>
    <abstract>Characteristic model phrasings like “delve” and em dashes were favored when rare, made ubiquitous by alignment, and are now penalized by readers as a marker of machine authorship: the style’s utility fell because it became prevalent, and it became prevalent because it was optimized. RLHF is misspecified for preferences whose value depends on how prevalent an output is in deployment. We formalize this as deployment-dependent utility U(y; D\ensuremathπ), distinguish it from distribution shift, define deployment-consistent policies and the performative gap, and prove that the deployment dependent component is unidentifiable from fixed-deployment preference data at any sample size, while deployment variation identifies it. In a controlled setting a static reward model becomes miscalibrated exactly where deployment concentrates, and RLHF and variational preference learning fail with and without oracle groups. Our central experiment instead elicits the utility from a frontier LLM judge over real text: the judge is robustly prevalence-averse under naturalistic exposure, the miscalibration and oracle failure reproduce, deployment-aware training under a misspecified deployment model is worst of all, and an approximately maximum-diversity policy beats every trained method, tying performative misspecification to RLHF’s documented diversity loss. The human prevalence-response remains unmeasured. Alignment under such preferences is a performative learning problem.</abstract>
    <identifier type="citekey">chinta-2026-performative</identifier>
    <location>
        <url>https://aclanthology.org/2026.plurvallm-1.3/</url>
    </location>
    <part>
        <date>2026-11</date>
        <extent unit="page">
            <start>24</start>
            <end>42</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Performative Alignment: Preference Learning under Deployment-Dependent Utility
%A Chinta, Tarun
%Y Xiong, Deyi
%Y Branco, António
%Y Zhang, Hongming
%Y Dong, Yue
%Y Wang, Benyou
%Y Zhang, Wenxuan
%Y Zhou, Li
%Y Zheng, Jingting
%S Proceedings of the First Workshop on Pluralistic Value Alignment of LLMs
%D 2026
%8 November
%I Association for Computational Linguistics
%C Hengqin, China
%@ 979-8-89176-453-8
%F chinta-2026-performative
%X Characteristic model phrasings like “delve” and em dashes were favored when rare, made ubiquitous by alignment, and are now penalized by readers as a marker of machine authorship: the style’s utility fell because it became prevalent, and it became prevalent because it was optimized. RLHF is misspecified for preferences whose value depends on how prevalent an output is in deployment. We formalize this as deployment-dependent utility U(y; D\ensuremathπ), distinguish it from distribution shift, define deployment-consistent policies and the performative gap, and prove that the deployment dependent component is unidentifiable from fixed-deployment preference data at any sample size, while deployment variation identifies it. In a controlled setting a static reward model becomes miscalibrated exactly where deployment concentrates, and RLHF and variational preference learning fail with and without oracle groups. Our central experiment instead elicits the utility from a frontier LLM judge over real text: the judge is robustly prevalence-averse under naturalistic exposure, the miscalibration and oracle failure reproduce, deployment-aware training under a misspecified deployment model is worst of all, and an approximately maximum-diversity policy beats every trained method, tying performative misspecification to RLHF’s documented diversity loss. The human prevalence-response remains unmeasured. Alignment under such preferences is a performative learning problem.
%U https://aclanthology.org/2026.plurvallm-1.3/
%P 24-42

Markdown (Informal)

[Performative Alignment: Preference Learning under Deployment-Dependent Utility](https://aclanthology.org/2026.plurvallm-1.3/) (Chinta, PlurVA-LLM 2026)

• Performative Alignment: Preference Learning under Deployment-Dependent Utility (Chinta, PlurVA-LLM 2026)

ACL

• Tarun Chinta. 2026. Performative Alignment: Preference Learning under Deployment-Dependent Utility. In Proceedings of the First Workshop on Pluralistic Value Alignment of LLMs, pages 24–42, Hengqin, China. Association for Computational Linguistics.