<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="hu-etal-2026-flip">
    <titleInfo>
        <title>The Flip-Test: A Behavioral Diagnostic for Multi-Agent Debate Failures</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Julia</namePart>
        <namePart type="family">Hu</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Alfred</namePart>
        <namePart type="family">Shen</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Kumar</namePart>
        <namePart type="family">Lakshmipathi</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2026-10</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the Seventh Workshop on Insights from Negative Results in NLP</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">João</namePart>
            <namePart type="family">Sedoc</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Raphael</namePart>
            <namePart type="family">Shu</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Aleksandr</namePart>
            <namePart type="family">Drozd</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Arjun</namePart>
            <namePart type="family">Akula</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Anna</namePart>
            <namePart type="family">Rogers</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Salvatore</namePart>
            <namePart type="family">Giorgi</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Adrian</namePart>
            <namePart type="family">Benton</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Budapest, Hungary</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
        <identifier type="isbn">979-8-89176-480-4</identifier>
    </relatedItem>
    <abstract>Multi-agent debate has been proposed to improve language model reasoning through adversarial interaction, yet empirical studies report that debate underperforms simpler baselines on open-ended reasoning (Choi et al., 2025). We propose a behavioral diagnostic—the flip-test—that instruments debate at the example level and measures how often, and in which direction, agents change their answers between initial and final rounds. On 2,417 MuSiQue examples with each of Llama 3.1 8B and Ministral 3 8B, flips are common (65–70% of debates). Our central result conditions on opportunity: a flip destroys an available correct answer\ensuremath\sim2.4×more often than it creates one (48.9% vs. 20.3% on Llama, 44.2% vs. 18.0% on Ministral; p 10\ensuremath-20). Unlike a raw directional split, this measure is invariant to how many correct answers were available, and it is statistically indistinguishable across the two model families. Three instruction-level interventions on a 100-example held-out set produce no improvement, suggesting simple prompt-level patches are insufficient. We separate this behavioral finding from mechanism: direct opponent-adoption is low (\ensuremath\sim10%), so the asymmetry is consistent with sycophancy but does not establish it. The signature is also scale-dependent—on three 70B-class models the flip rate roughly halves and debate turns net-positive—placing the harmful regime in the 8B capacity band rather than in debate itself. We frame the flip-test as a cheap behavioural probe for locating that regime.</abstract>
    <identifier type="citekey">hu-etal-2026-flip</identifier>
    <location>
        <url>https://aclanthology.org/2026.insights-1.8/</url>
    </location>
    <part>
        <date>2026-10</date>
        <extent unit="page">
            <start>70</start>
            <end>75</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T The Flip-Test: A Behavioral Diagnostic for Multi-Agent Debate Failures
%A Hu, Julia
%A Shen, Alfred
%A Lakshmipathi, Kumar
%Y Sedoc, João
%Y Shu, Raphael
%Y Drozd, Aleksandr
%Y Akula, Arjun
%Y Rogers, Anna
%Y Giorgi, Salvatore
%Y Benton, Adrian
%S Proceedings of the Seventh Workshop on Insights from Negative Results in NLP
%D 2026
%8 October
%I Association for Computational Linguistics
%C Budapest, Hungary
%@ 979-8-89176-480-4
%F hu-etal-2026-flip
%X Multi-agent debate has been proposed to improve language model reasoning through adversarial interaction, yet empirical studies report that debate underperforms simpler baselines on open-ended reasoning (Choi et al., 2025). We propose a behavioral diagnostic—the flip-test—that instruments debate at the example level and measures how often, and in which direction, agents change their answers between initial and final rounds. On 2,417 MuSiQue examples with each of Llama 3.1 8B and Ministral 3 8B, flips are common (65–70% of debates). Our central result conditions on opportunity: a flip destroys an available correct answer\ensuremath\sim2.4×more often than it creates one (48.9% vs. 20.3% on Llama, 44.2% vs. 18.0% on Ministral; p 10\ensuremath-20). Unlike a raw directional split, this measure is invariant to how many correct answers were available, and it is statistically indistinguishable across the two model families. Three instruction-level interventions on a 100-example held-out set produce no improvement, suggesting simple prompt-level patches are insufficient. We separate this behavioral finding from mechanism: direct opponent-adoption is low (\ensuremath\sim10%), so the asymmetry is consistent with sycophancy but does not establish it. The signature is also scale-dependent—on three 70B-class models the flip rate roughly halves and debate turns net-positive—placing the harmful regime in the 8B capacity band rather than in debate itself. We frame the flip-test as a cheap behavioural probe for locating that regime.
%U https://aclanthology.org/2026.insights-1.8/
%P 70-75

Markdown (Informal)

[The Flip-Test: A Behavioral Diagnostic for Multi-Agent Debate Failures](https://aclanthology.org/2026.insights-1.8/) (Hu et al., insights 2026)

• The Flip-Test: A Behavioral Diagnostic for Multi-Agent Debate Failures (Hu et al., insights 2026)

ACL

• Julia Hu, Alfred Shen, and Kumar Lakshmipathi. 2026. The Flip-Test: A Behavioral Diagnostic for Multi-Agent Debate Failures. In Proceedings of the Seventh Workshop on Insights from Negative Results in NLP, pages 70–75, Budapest, Hungary. Association for Computational Linguistics.