<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="chae-etal-2026-self">
    <titleInfo>
        <title>Self- and Other-Labels Induce Bidirectional Bias in LLM Judges</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Songeun</namePart>
        <namePart type="family">Chae</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Min</namePart>
        <namePart type="family">Kim</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Donghoon</namePart>
        <namePart type="family">Jung</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Seojin</namePart>
        <namePart type="family">Choi</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Seohyon</namePart>
        <namePart type="family">Jung</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2026-10</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Findings of the Association for Computational Linguistics: EMNLP 2026</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Hung-Yi</namePart>
            <namePart type="family">Lee</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Sunipa</namePart>
            <namePart type="family">Dev</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Jessy</namePart>
            <namePart type="family">Li</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Desmond</namePart>
            <namePart type="family">Elliott</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Budapest, Hungary</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
        <identifier type="isbn">979-8-89176-462-0</identifier>
    </relatedItem>
    <abstract>As LLM-as-a-judge becomes increasingly widespread, self-preference—the tendency of a judge to favor its own outputs—raises growing concerns about evaluation reliability. However, this bias has been studied predominantly on generated text, where stylistic features and response quality are inevitably conflated. As a result, existing measurements cannot separate genuine self-preference from these confounds. We address this limitation by changing the object of evaluation: instead of judging generated text, ten LLMs assess sets of narrative constraints selected from a shared pool, which carry no stylistic fingerprint yet retain a recoverable model-specific signature. Two experiments on this task yield complementary findings. Under blind evaluation, self-preference disappears, with a small effect remaining in the opposite direction once selection quality and judge severity are controlled. Under matched quality, however, self- and other-labels alone—without naming any model—shift scores bidirectionally. LLM judges inflate scores for self-labeled selections and deflate those for other-labeled ones regardless of the selection’s actual source. We make two contributions: 1) authorship attribution is a distinct driver of evaluation bias, and 2) ground-truth-free tasks can serve as controlled instruments for studying LLM judge behavior.</abstract>
    <identifier type="citekey">chae-etal-2026-self</identifier>
    <identifier type="doi">10.18653/2026.findings-emnlp.1571</identifier>
    <location>
        <url>https://aclanthology.org/2026.findings-emnlp.1571/</url>
    </location>
    <part>
        <date>2026-10</date>
        <extent unit="page">
            <start>33466</start>
            <end>33482</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Self- and Other-Labels Induce Bidirectional Bias in LLM Judges
%A Chae, Songeun
%A Kim, Min
%A Jung, Donghoon
%A Choi, Seojin
%A Jung, Seohyon
%Y Lee, Hung-Yi
%Y Dev, Sunipa
%Y Li, Jessy
%Y Elliott, Desmond
%S Findings of the Association for Computational Linguistics: EMNLP 2026
%D 2026
%8 October
%I Association for Computational Linguistics
%C Budapest, Hungary
%@ 979-8-89176-462-0
%F chae-etal-2026-self
%X As LLM-as-a-judge becomes increasingly widespread, self-preference—the tendency of a judge to favor its own outputs—raises growing concerns about evaluation reliability. However, this bias has been studied predominantly on generated text, where stylistic features and response quality are inevitably conflated. As a result, existing measurements cannot separate genuine self-preference from these confounds. We address this limitation by changing the object of evaluation: instead of judging generated text, ten LLMs assess sets of narrative constraints selected from a shared pool, which carry no stylistic fingerprint yet retain a recoverable model-specific signature. Two experiments on this task yield complementary findings. Under blind evaluation, self-preference disappears, with a small effect remaining in the opposite direction once selection quality and judge severity are controlled. Under matched quality, however, self- and other-labels alone—without naming any model—shift scores bidirectionally. LLM judges inflate scores for self-labeled selections and deflate those for other-labeled ones regardless of the selection’s actual source. We make two contributions: 1) authorship attribution is a distinct driver of evaluation bias, and 2) ground-truth-free tasks can serve as controlled instruments for studying LLM judge behavior.
%R 10.18653/2026.findings-emnlp.1571
%U https://aclanthology.org/2026.findings-emnlp.1571/
%U https://doi.org/10.18653/2026.findings-emnlp.1571
%P 33466-33482

Markdown (Informal)

[Self- and Other-Labels Induce Bidirectional Bias in LLM Judges](https://aclanthology.org/2026.findings-emnlp.1571/) (Chae et al., Findings 2026)

• Self- and Other-Labels Induce Bidirectional Bias in LLM Judges (Chae et al., Findings 2026)

ACL

• Songeun Chae, Min Kim, Donghoon Jung, Seojin Choi, and Seohyon Jung. 2026. Self- and Other-Labels Induce Bidirectional Bias in LLM Judges. In Findings of the Association for Computational Linguistics: EMNLP 2026, pages 33466–33482, Budapest, Hungary. Association for Computational Linguistics.