<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="hager-etal-2026-inverse">
    <titleInfo>
        <title>Inverse Turing Bench: Evaluating Language Models as Judges of Human vs. AI Dialogue</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">William</namePart>
        <namePart type="family">Hager</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Ishika</namePart>
        <namePart type="family">Rathi</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Masum</namePart>
        <namePart type="family">Hasan</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Cameron</namePart>
        <namePart type="given">Robert</namePart>
        <namePart type="family">Jones</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2026-10</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the 3rd Workshop on Language Understanding in the Human-Machine Era (LUHME 2026)</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Rui</namePart>
            <namePart type="family">Sousa-Silva</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Henrique</namePart>
            <namePart type="family">Lopes Cardoso</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Maarit</namePart>
            <namePart type="family">Koponen</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Antonio</namePart>
            <namePart type="family">Pareja-Lora</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Budapest, Hungary</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
        <identifier type="isbn">979-8-89176-471-2</identifier>
    </relatedItem>
    <abstract>As AI systems integrate into online spaces, differentiating them from humans in conversations is increasingly important. We present INVERSE TURING BENCH, a benchmark that evaluates LLMs and other models on their ability to differentiate humans and AI in multi-turn text. The benchmark provides a collection of paired dialogue transcripts, wherein one dialogue is between two humans and the other is between a human and an AI. The task is to correctly identify which dialogue is human-only vs. human-AI. We evaluated a preliminary set of models against this benchmark, and found that GPTZero, Claude Opus-4.6, and GPT-5.5 achieve the highest accuracy: 89.41%, 77.92%, and 75.94% respectively. Our results suggest that statistical approaches to detection have semantic blind spots, but semantic approaches are susceptible to persona-prompting. Our work speaks to the Inverse Turing Test and motivates human-AI differentiation as a critical capability for AI systems. Our live benchmark can be found at https://huggingface.co/spaces/roc-hci/Inverse-Turing-Bench-Leaderboard</abstract>
    <identifier type="citekey">hager-etal-2026-inverse</identifier>
    <location>
        <url>https://aclanthology.org/2026.luhme-1.25/</url>
    </location>
    <part>
        <date>2026-10</date>
        <extent unit="page">
            <start>378</start>
            <end>388</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Inverse Turing Bench: Evaluating Language Models as Judges of Human vs. AI Dialogue
%A Hager, William
%A Rathi, Ishika
%A Hasan, Masum
%A Jones, Cameron Robert
%Y Sousa-Silva, Rui
%Y Lopes Cardoso, Henrique
%Y Koponen, Maarit
%Y Pareja-Lora, Antonio
%S Proceedings of the 3rd Workshop on Language Understanding in the Human-Machine Era (LUHME 2026)
%D 2026
%8 October
%I Association for Computational Linguistics
%C Budapest, Hungary
%@ 979-8-89176-471-2
%F hager-etal-2026-inverse
%X As AI systems integrate into online spaces, differentiating them from humans in conversations is increasingly important. We present INVERSE TURING BENCH, a benchmark that evaluates LLMs and other models on their ability to differentiate humans and AI in multi-turn text. The benchmark provides a collection of paired dialogue transcripts, wherein one dialogue is between two humans and the other is between a human and an AI. The task is to correctly identify which dialogue is human-only vs. human-AI. We evaluated a preliminary set of models against this benchmark, and found that GPTZero, Claude Opus-4.6, and GPT-5.5 achieve the highest accuracy: 89.41%, 77.92%, and 75.94% respectively. Our results suggest that statistical approaches to detection have semantic blind spots, but semantic approaches are susceptible to persona-prompting. Our work speaks to the Inverse Turing Test and motivates human-AI differentiation as a critical capability for AI systems. Our live benchmark can be found at https://huggingface.co/spaces/roc-hci/Inverse-Turing-Bench-Leaderboard
%U https://aclanthology.org/2026.luhme-1.25/
%P 378-388

Markdown (Informal)

[Inverse Turing Bench: Evaluating Language Models as Judges of Human vs. AI Dialogue](https://aclanthology.org/2026.luhme-1.25/) (Hager et al., LUHME 2026)

• Inverse Turing Bench: Evaluating Language Models as Judges of Human vs. AI Dialogue (Hager et al., LUHME 2026)

ACL

• William Hager, Ishika Rathi, Masum Hasan, and Cameron Robert Jones. 2026. Inverse Turing Bench: Evaluating Language Models as Judges of Human vs. AI Dialogue. In Proceedings of the 3rd Workshop on Language Understanding in the Human-Machine Era (LUHME 2026), pages 378–388, Budapest, Hungary. Association for Computational Linguistics.