<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="niu-etal-2026-beyond">
    <titleInfo>
        <title>Beyond Detection: Evaluating Fallacy Awareness of LLMs in Interactive Scenarios</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Conghui</namePart>
        <namePart type="family">Niu</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Ningxin</namePart>
        <namePart type="family">Wu</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Ziran</namePart>
        <namePart type="family">Zhao</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Dong</namePart>
        <namePart type="family">Yu</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Chen</namePart>
        <namePart type="family">Kang</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Pengyuan</namePart>
        <namePart type="family">Liu</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2026-07</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Maria</namePart>
            <namePart type="family">Liakata</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Viviane</namePart>
            <namePart type="given">P</namePart>
            <namePart type="family">Moreira</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Jiajun</namePart>
            <namePart type="family">Zhang</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">David</namePart>
            <namePart type="family">Jurgens</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">San Diego, California, United States</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
        <identifier type="isbn">979-8-89176-390-6</identifier>
    </relatedItem>
    <abstract>Large Language Models (LLMs) often fail to recognize fallacious reasoning in real-world interactions, despite strong performance on static fallacy detection tasks. We define this ability as fallacy awareness, the capacity to autonomously perceive and resist fallacies in dynamic, pragmatic contexts. To study this, we introduce ISFallacy, a large-scale Chinese benchmark of 50K interactive scenarios spanning six fallacy types, five social interaction settings, diverse role relationships, and personality traits. We further propose FATE, a two-stage evaluation framework that assesses fallacy awareness without explicit cues, combining natural dialogue responses and reasoning-based decisions. Experiments on five representative LLMs reveal a substantial gap between fallacy classification and awareness, with models particularly vulnerable to emotion-driven fallacies and scenarios involving cooperative or trust-based relationships. Deeper analysis uncovers a cognition–behavior gap and fragile internal representations underlying awareness failures. Our work establishes a foundation for evaluating and enhancing the robustness of LLMs against fallacious reasoning in interactive settings.</abstract>
    <identifier type="citekey">niu-etal-2026-beyond</identifier>
    <location>
        <url>https://aclanthology.org/2026.acl-long.59/</url>
    </location>
    <part>
        <date>2026-07</date>
        <extent unit="page">
            <start>1331</start>
            <end>1352</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Beyond Detection: Evaluating Fallacy Awareness of LLMs in Interactive Scenarios
%A Niu, Conghui
%A Wu, Ningxin
%A Zhao, Ziran
%A Yu, Dong
%A Kang, Chen
%A Liu, Pengyuan
%Y Liakata, Maria
%Y Moreira, Viviane P.
%Y Zhang, Jiajun
%Y Jurgens, David
%S Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
%D 2026
%8 July
%I Association for Computational Linguistics
%C San Diego, California, United States
%@ 979-8-89176-390-6
%F niu-etal-2026-beyond
%X Large Language Models (LLMs) often fail to recognize fallacious reasoning in real-world interactions, despite strong performance on static fallacy detection tasks. We define this ability as fallacy awareness, the capacity to autonomously perceive and resist fallacies in dynamic, pragmatic contexts. To study this, we introduce ISFallacy, a large-scale Chinese benchmark of 50K interactive scenarios spanning six fallacy types, five social interaction settings, diverse role relationships, and personality traits. We further propose FATE, a two-stage evaluation framework that assesses fallacy awareness without explicit cues, combining natural dialogue responses and reasoning-based decisions. Experiments on five representative LLMs reveal a substantial gap between fallacy classification and awareness, with models particularly vulnerable to emotion-driven fallacies and scenarios involving cooperative or trust-based relationships. Deeper analysis uncovers a cognition–behavior gap and fragile internal representations underlying awareness failures. Our work establishes a foundation for evaluating and enhancing the robustness of LLMs against fallacious reasoning in interactive settings.
%U https://aclanthology.org/2026.acl-long.59/
%P 1331-1352

Markdown (Informal)

[Beyond Detection: Evaluating Fallacy Awareness of LLMs in Interactive Scenarios](https://aclanthology.org/2026.acl-long.59/) (Niu et al., ACL 2026)

• Beyond Detection: Evaluating Fallacy Awareness of LLMs in Interactive Scenarios (Niu et al., ACL 2026)

ACL

• Conghui Niu, Ningxin Wu, Ziran Zhao, Dong Yu, Chen Kang, and Pengyuan Liu. 2026. Beyond Detection: Evaluating Fallacy Awareness of LLMs in Interactive Scenarios. In Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1331–1352, San Diego, California, United States. Association for Computational Linguistics.