<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="leidinger-etal-2024-llms">
    <titleInfo>
        <title>Are LLMs classical or nonmonotonic reasoners? Lessons from generics</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Alina</namePart>
        <namePart type="family">Leidinger</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Robert</namePart>
        <namePart type="family">Van Rooij</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Ekaterina</namePart>
        <namePart type="family">Shutova</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2024-08</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Lun-Wei</namePart>
            <namePart type="family">Ku</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Andre</namePart>
            <namePart type="family">Martins</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Vivek</namePart>
            <namePart type="family">Srikumar</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Bangkok, Thailand</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>Recent scholarship on reasoning in LLMs has supplied evidence of impressive performance and flexible adaptation to machine generated or human critique. Nonmonotonic reasoning, crucial to human cognition for navigating the real world, remains a challenging, yet understudied task. In this work, we study nonmonotonic reasoning capabilities of seven state-of-the-art LLMs in one abstract and one commonsense reasoning task featuring generics, such as ‘Birds fly’, and exceptions, ‘Penguins don’t fly’ (see Fig. 1). While LLMs exhibit reasoning patterns in accordance with human nonmonotonic reasoning abilities, they fail to maintain stable beliefs on truth conditions of generics at the addition of supporting examples (‘Owls fly’) or unrelated information (‘Lions have manes’).Our findings highlight pitfalls in attributing human reasoning behaviours to LLMs as long as consistent reasoning remains elusive.</abstract>
    <identifier type="citekey">leidinger-etal-2024-llms</identifier>
    <identifier type="doi">10.18653/v1/2024.acl-short.51</identifier>
    <location>
        <url>https://aclanthology.org/2024.acl-short.51</url>
    </location>
    <part>
        <date>2024-08</date>
        <extent unit="page">
            <start>558</start>
            <end>573</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Are LLMs classical or nonmonotonic reasoners? Lessons from generics
%A Leidinger, Alina
%A Van Rooij, Robert
%A Shutova, Ekaterina
%Y Ku, Lun-Wei
%Y Martins, Andre
%Y Srikumar, Vivek
%S Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)
%D 2024
%8 August
%I Association for Computational Linguistics
%C Bangkok, Thailand
%F leidinger-etal-2024-llms
%X Recent scholarship on reasoning in LLMs has supplied evidence of impressive performance and flexible adaptation to machine generated or human critique. Nonmonotonic reasoning, crucial to human cognition for navigating the real world, remains a challenging, yet understudied task. In this work, we study nonmonotonic reasoning capabilities of seven state-of-the-art LLMs in one abstract and one commonsense reasoning task featuring generics, such as ‘Birds fly’, and exceptions, ‘Penguins don’t fly’ (see Fig. 1). While LLMs exhibit reasoning patterns in accordance with human nonmonotonic reasoning abilities, they fail to maintain stable beliefs on truth conditions of generics at the addition of supporting examples (‘Owls fly’) or unrelated information (‘Lions have manes’).Our findings highlight pitfalls in attributing human reasoning behaviours to LLMs as long as consistent reasoning remains elusive.
%R 10.18653/v1/2024.acl-short.51
%U https://aclanthology.org/2024.acl-short.51
%U https://doi.org/10.18653/v1/2024.acl-short.51
%P 558-573

Markdown (Informal)

[Are LLMs classical or nonmonotonic reasoners? Lessons from generics](https://aclanthology.org/2024.acl-short.51) (Leidinger et al., ACL 2024)

• Are LLMs classical or nonmonotonic reasoners? Lessons from generics (Leidinger et al., ACL 2024)

ACL

• Alina Leidinger, Robert Van Rooij, and Ekaterina Shutova. 2024. Are LLMs classical or nonmonotonic reasoners? Lessons from generics. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 558–573, Bangkok, Thailand. Association for Computational Linguistics.