<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="zhou-etal-2022-hierarchical-control">
    <titleInfo>
        <title>Hierarchical Control of Situated Agents through Natural Language</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Shuyan</namePart>
        <namePart type="family">Zhou</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Pengcheng</namePart>
        <namePart type="family">Yin</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Graham</namePart>
        <namePart type="family">Neubig</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2022-07</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the Workshop on Structured and Unstructured Knowledge Integration (SUKI)</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Wenhu</namePart>
            <namePart type="family">Chen</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Xinyun</namePart>
            <namePart type="family">Chen</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Zhiyu</namePart>
            <namePart type="family">Chen</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Ziyu</namePart>
            <namePart type="family">Yao</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Michihiro</namePart>
            <namePart type="family">Yasunaga</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Tao</namePart>
            <namePart type="family">Yu</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Rui</namePart>
            <namePart type="family">Zhang</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Seattle, USA</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>When humans perform a particular task, they do so hierarchically: splitting higher-level tasks into smaller sub-tasks. However, most works on natural language (NL) command of situated agents have treated the procedures to be executed as flat sequences of simple actions, or any hierarchies of procedures have been shallow at best. In this paper, we propose a formalism of procedures as programs, a method for representing hierarchical procedural knowledge for agent command and control aimed at enabling easy application to various scenarios. We further propose a modeling paradigm of hierarchical modular networks, which consist of a planner and reactors that convert NL intents to predictions of executable programs and probe the environment for information necessary to complete the program execution. We instantiate this framework on the IQA and ALFRED datasets for NL instruction following. Our model outperforms reactive baselines by a large margin on both datasets. We also demonstrate that our framework is more data-efficient, and that it allows for fast iterative development.</abstract>
    <identifier type="citekey">zhou-etal-2022-hierarchical-control</identifier>
    <identifier type="doi">10.18653/v1/2022.suki-1.8</identifier>
    <location>
        <url>https://aclanthology.org/2022.suki-1.8</url>
    </location>
    <part>
        <date>2022-07</date>
        <extent unit="page">
            <start>67</start>
            <end>84</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Hierarchical Control of Situated Agents through Natural Language
%A Zhou, Shuyan
%A Yin, Pengcheng
%A Neubig, Graham
%Y Chen, Wenhu
%Y Chen, Xinyun
%Y Chen, Zhiyu
%Y Yao, Ziyu
%Y Yasunaga, Michihiro
%Y Yu, Tao
%Y Zhang, Rui
%S Proceedings of the Workshop on Structured and Unstructured Knowledge Integration (SUKI)
%D 2022
%8 July
%I Association for Computational Linguistics
%C Seattle, USA
%F zhou-etal-2022-hierarchical-control
%X When humans perform a particular task, they do so hierarchically: splitting higher-level tasks into smaller sub-tasks. However, most works on natural language (NL) command of situated agents have treated the procedures to be executed as flat sequences of simple actions, or any hierarchies of procedures have been shallow at best. In this paper, we propose a formalism of procedures as programs, a method for representing hierarchical procedural knowledge for agent command and control aimed at enabling easy application to various scenarios. We further propose a modeling paradigm of hierarchical modular networks, which consist of a planner and reactors that convert NL intents to predictions of executable programs and probe the environment for information necessary to complete the program execution. We instantiate this framework on the IQA and ALFRED datasets for NL instruction following. Our model outperforms reactive baselines by a large margin on both datasets. We also demonstrate that our framework is more data-efficient, and that it allows for fast iterative development.
%R 10.18653/v1/2022.suki-1.8
%U https://aclanthology.org/2022.suki-1.8
%U https://doi.org/10.18653/v1/2022.suki-1.8
%P 67-84

Markdown (Informal)

[Hierarchical Control of Situated Agents through Natural Language](https://aclanthology.org/2022.suki-1.8) (Zhou et al., SUKI 2022)

• Hierarchical Control of Situated Agents through Natural Language (Zhou et al., SUKI 2022)

ACL

• Shuyan Zhou, Pengcheng Yin, and Graham Neubig. 2022. Hierarchical Control of Situated Agents through Natural Language. In Proceedings of the Workshop on Structured and Unstructured Knowledge Integration (SUKI), pages 67–84, Seattle, USA. Association for Computational Linguistics.