<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="moniri-etal-2025-evaluating">
    <titleInfo>
        <title>Evaluating the Performance of Large Language Models via Debates</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Behrad</namePart>
        <namePart type="family">Moniri</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Hamed</namePart>
        <namePart type="family">Hassani</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Edgar</namePart>
        <namePart type="family">Dobriban</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2025-04</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Findings of the Association for Computational Linguistics: NAACL 2025</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Luis</namePart>
            <namePart type="family">Chiruzzo</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Alan</namePart>
            <namePart type="family">Ritter</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Lu</namePart>
            <namePart type="family">Wang</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Albuquerque, New Mexico</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
        <identifier type="isbn">979-8-89176-195-7</identifier>
    </relatedItem>
    <abstract>Large Language Models (LLMs) are rapidly evolving and impacting various fields, necessitating the development of effective methods to evaluate and compare their performance. Most current approaches for performance evaluation are either based on fixed, domain-specific questions that lack the flexibility required in many real-world applications, or rely on human input, making them unscalable. To address these issues, we propose an automated benchmarking framework based on debates between LLMs, judged by another LLM. This method assesses not only domain knowledge, but also skills such as argumentative reasoning and inconsistency recognition. We evaluate the performance of various state-of-the-art LLMs using the debate framework and achieve rankings that align closely with popular rankings based on human input, eliminating the need for costly human crowdsourcing.</abstract>
    <identifier type="citekey">moniri-etal-2025-evaluating</identifier>
    <identifier type="doi">10.18653/v1/2025.findings-naacl.109</identifier>
    <location>
        <url>https://aclanthology.org/2025.findings-naacl.109/</url>
    </location>
    <part>
        <date>2025-04</date>
        <extent unit="page">
            <start>2040</start>
            <end>2075</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Evaluating the Performance of Large Language Models via Debates
%A Moniri, Behrad
%A Hassani, Hamed
%A Dobriban, Edgar
%Y Chiruzzo, Luis
%Y Ritter, Alan
%Y Wang, Lu
%S Findings of the Association for Computational Linguistics: NAACL 2025
%D 2025
%8 April
%I Association for Computational Linguistics
%C Albuquerque, New Mexico
%@ 979-8-89176-195-7
%F moniri-etal-2025-evaluating
%X Large Language Models (LLMs) are rapidly evolving and impacting various fields, necessitating the development of effective methods to evaluate and compare their performance. Most current approaches for performance evaluation are either based on fixed, domain-specific questions that lack the flexibility required in many real-world applications, or rely on human input, making them unscalable. To address these issues, we propose an automated benchmarking framework based on debates between LLMs, judged by another LLM. This method assesses not only domain knowledge, but also skills such as argumentative reasoning and inconsistency recognition. We evaluate the performance of various state-of-the-art LLMs using the debate framework and achieve rankings that align closely with popular rankings based on human input, eliminating the need for costly human crowdsourcing.
%R 10.18653/v1/2025.findings-naacl.109
%U https://aclanthology.org/2025.findings-naacl.109/
%U https://doi.org/10.18653/v1/2025.findings-naacl.109
%P 2040-2075

Markdown (Informal)

[Evaluating the Performance of Large Language Models via Debates](https://aclanthology.org/2025.findings-naacl.109/) (Moniri et al., Findings 2025)

• Evaluating the Performance of Large Language Models via Debates (Moniri et al., Findings 2025)

ACL

• Behrad Moniri, Hamed Hassani, and Edgar Dobriban. 2025. Evaluating the Performance of Large Language Models via Debates. In Findings of the Association for Computational Linguistics: NAACL 2025, pages 2040–2075, Albuquerque, New Mexico. Association for Computational Linguistics.