<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="adak-etal-2024-text2afford">
    <titleInfo>
        <title>Text2Afford: Probing Object Affordance Prediction abilities of Language Models solely from Text</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Sayantan</namePart>
        <namePart type="family">Adak</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Daivik</namePart>
        <namePart type="family">Agrawal</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Animesh</namePart>
        <namePart type="family">Mukherjee</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Somak</namePart>
        <namePart type="family">Aditya</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2024-11</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the 28th Conference on Computational Natural Language Learning</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Libby</namePart>
            <namePart type="family">Barak</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Malihe</namePart>
            <namePart type="family">Alikhani</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Miami, FL, USA</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>We investigate the knowledge of object affordances in pre-trained language models (LMs) and pre-trained Vision-Language models (VLMs).A growing body of literature shows that PTLMs fail inconsistently and non-intuitively, demonstrating a lack of reasoning and grounding. To take a first step toward quantifying the effect of grounding (or lack thereof), we curate a novel and comprehensive dataset of object affordances – Text2Afford, characterized by 15 affordance classes. Unlike affordance datasets collected in vision and language domains, we annotate in-the-wild sentences with objects and affordances. Experimental results reveal that PTLMs exhibit limited reasoning abilities when it comes to uncommon object affordances. We also observe that pre-trained VLMs do not necessarily capture object affordances effectively. Through few-shot fine-tuning, we demonstrate improvement in affordance knowledge in PTLMs and VLMs. Our research contributes a novel dataset for language grounding tasks, and presents insights into LM capabilities, advancing the understanding of object affordances.</abstract>
    <identifier type="citekey">adak-etal-2024-text2afford</identifier>
    <location>
        <url>https://aclanthology.org/2024.conll-1.27</url>
    </location>
    <part>
        <date>2024-11</date>
        <extent unit="page">
            <start>342</start>
            <end>364</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Text2Afford: Probing Object Affordance Prediction abilities of Language Models solely from Text
%A Adak, Sayantan
%A Agrawal, Daivik
%A Mukherjee, Animesh
%A Aditya, Somak
%Y Barak, Libby
%Y Alikhani, Malihe
%S Proceedings of the 28th Conference on Computational Natural Language Learning
%D 2024
%8 November
%I Association for Computational Linguistics
%C Miami, FL, USA
%F adak-etal-2024-text2afford
%X We investigate the knowledge of object affordances in pre-trained language models (LMs) and pre-trained Vision-Language models (VLMs).A growing body of literature shows that PTLMs fail inconsistently and non-intuitively, demonstrating a lack of reasoning and grounding. To take a first step toward quantifying the effect of grounding (or lack thereof), we curate a novel and comprehensive dataset of object affordances – Text2Afford, characterized by 15 affordance classes. Unlike affordance datasets collected in vision and language domains, we annotate in-the-wild sentences with objects and affordances. Experimental results reveal that PTLMs exhibit limited reasoning abilities when it comes to uncommon object affordances. We also observe that pre-trained VLMs do not necessarily capture object affordances effectively. Through few-shot fine-tuning, we demonstrate improvement in affordance knowledge in PTLMs and VLMs. Our research contributes a novel dataset for language grounding tasks, and presents insights into LM capabilities, advancing the understanding of object affordances.
%U https://aclanthology.org/2024.conll-1.27
%P 342-364

Markdown (Informal)

[Text2Afford: Probing Object Affordance Prediction abilities of Language Models solely from Text](https://aclanthology.org/2024.conll-1.27) (Adak et al., CoNLL 2024)

• Text2Afford: Probing Object Affordance Prediction abilities of Language Models solely from Text (Adak et al., CoNLL 2024)

ACL

• Sayantan Adak, Daivik Agrawal, Animesh Mukherjee, and Somak Aditya. 2024. Text2Afford: Probing Object Affordance Prediction abilities of Language Models solely from Text. In Proceedings of the 28th Conference on Computational Natural Language Learning, pages 342–364, Miami, FL, USA. Association for Computational Linguistics.