<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="liu-etal-2023-picture">
    <titleInfo>
        <title>A Picture is Worth a Thousand Words: Language Models Plan from Pixels</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Anthony</namePart>
        <namePart type="family">Liu</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Lajanugen</namePart>
        <namePart type="family">Logeswaran</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Sungryull</namePart>
        <namePart type="family">Sohn</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Honglak</namePart>
        <namePart type="family">Lee</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2023-12</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Houda</namePart>
            <namePart type="family">Bouamor</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Juan</namePart>
            <namePart type="family">Pino</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Kalika</namePart>
            <namePart type="family">Bali</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Singapore</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>Planning is an important capability of artificial agents that perform long-horizon tasks in real-world environments. In this work, we explore the use of pre-trained language models (PLMs) to reason about plan sequences from text instructions in embodied visual environments. Prior PLM based approaches for planning either assume observations are available in the form of text by a captioning model, reason about plans from the instruction alone, or incorporate information about the visual environment in limited ways (such as a pre-trained affordance function). In contrast, we show that the PLM can accurately plan even when observations are directly encoded as input prompts for the PLM. We show this simple approach outperforms prior approaches in experiments on the ALFWorld and VirtualHome benchmarks.</abstract>
    <identifier type="citekey">liu-etal-2023-picture</identifier>
    <identifier type="doi">10.18653/v1/2023.emnlp-main.1025</identifier>
    <location>
        <url>https://aclanthology.org/2023.emnlp-main.1025</url>
    </location>
    <part>
        <date>2023-12</date>
        <extent unit="page">
            <start>16450</start>
            <end>16459</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T A Picture is Worth a Thousand Words: Language Models Plan from Pixels
%A Liu, Anthony
%A Logeswaran, Lajanugen
%A Sohn, Sungryull
%A Lee, Honglak
%Y Bouamor, Houda
%Y Pino, Juan
%Y Bali, Kalika
%S Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing
%D 2023
%8 December
%I Association for Computational Linguistics
%C Singapore
%F liu-etal-2023-picture
%X Planning is an important capability of artificial agents that perform long-horizon tasks in real-world environments. In this work, we explore the use of pre-trained language models (PLMs) to reason about plan sequences from text instructions in embodied visual environments. Prior PLM based approaches for planning either assume observations are available in the form of text by a captioning model, reason about plans from the instruction alone, or incorporate information about the visual environment in limited ways (such as a pre-trained affordance function). In contrast, we show that the PLM can accurately plan even when observations are directly encoded as input prompts for the PLM. We show this simple approach outperforms prior approaches in experiments on the ALFWorld and VirtualHome benchmarks.
%R 10.18653/v1/2023.emnlp-main.1025
%U https://aclanthology.org/2023.emnlp-main.1025
%U https://doi.org/10.18653/v1/2023.emnlp-main.1025
%P 16450-16459

Markdown (Informal)

[A Picture is Worth a Thousand Words: Language Models Plan from Pixels](https://aclanthology.org/2023.emnlp-main.1025) (Liu et al., EMNLP 2023)

• A Picture is Worth a Thousand Words: Language Models Plan from Pixels (Liu et al., EMNLP 2023)

ACL

• Anthony Liu, Lajanugen Logeswaran, Sungryull Sohn, and Honglak Lee. 2023. A Picture is Worth a Thousand Words: Language Models Plan from Pixels. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 16450–16459, Singapore. Association for Computational Linguistics.