<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="dou-peng-2022-foam">
    <titleInfo>
        <title>FOAM: A Follower-aware Speaker Model For Vision-and-Language Navigation</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Zi-Yi</namePart>
        <namePart type="family">Dou</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Nanyun</namePart>
        <namePart type="family">Peng</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2022-07</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Marine</namePart>
            <namePart type="family">Carpuat</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Marie-Catherine</namePart>
            <namePart type="family">de Marneffe</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Ivan</namePart>
            <namePart type="given">Vladimir</namePart>
            <namePart type="family">Meza Ruiz</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Seattle, United States</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>The speaker-follower models have proven to be effective in vision-and-language navigation, where a speaker model is used to synthesize new instructions to augment the training data for a follower navigation model. However, in previous work, the speaker model is follower-agnostic and fails to take the state of the follower into consideration. In this paper, we present FOAM, a FOllower-Aware speaker Model that is constantly updated given the follower feedback, so that the generated instructions can be more suitable to the current learning state of the follower. Specifically, we optimize the speaker using a bi-level optimization framework and obtain its training signals by evaluating the follower on labeled data. Experimental results on the Room-to-Room and Room-across-Room datasets demonstrate that our methods can outperform strong baseline models across settings. Analyses also reveal that our generated instructions are of higher quality than the baselines.</abstract>
    <identifier type="citekey">dou-peng-2022-foam</identifier>
    <identifier type="doi">10.18653/v1/2022.naacl-main.322</identifier>
    <location>
        <url>https://aclanthology.org/2022.naacl-main.322</url>
    </location>
    <part>
        <date>2022-07</date>
        <extent unit="page">
            <start>4332</start>
            <end>4340</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T FOAM: A Follower-aware Speaker Model For Vision-and-Language Navigation
%A Dou, Zi-Yi
%A Peng, Nanyun
%Y Carpuat, Marine
%Y de Marneffe, Marie-Catherine
%Y Meza Ruiz, Ivan Vladimir
%S Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies
%D 2022
%8 July
%I Association for Computational Linguistics
%C Seattle, United States
%F dou-peng-2022-foam
%X The speaker-follower models have proven to be effective in vision-and-language navigation, where a speaker model is used to synthesize new instructions to augment the training data for a follower navigation model. However, in previous work, the speaker model is follower-agnostic and fails to take the state of the follower into consideration. In this paper, we present FOAM, a FOllower-Aware speaker Model that is constantly updated given the follower feedback, so that the generated instructions can be more suitable to the current learning state of the follower. Specifically, we optimize the speaker using a bi-level optimization framework and obtain its training signals by evaluating the follower on labeled data. Experimental results on the Room-to-Room and Room-across-Room datasets demonstrate that our methods can outperform strong baseline models across settings. Analyses also reveal that our generated instructions are of higher quality than the baselines.
%R 10.18653/v1/2022.naacl-main.322
%U https://aclanthology.org/2022.naacl-main.322
%U https://doi.org/10.18653/v1/2022.naacl-main.322
%P 4332-4340

Markdown (Informal)

[FOAM: A Follower-aware Speaker Model For Vision-and-Language Navigation](https://aclanthology.org/2022.naacl-main.322) (Dou & Peng, NAACL 2022)

• FOAM: A Follower-aware Speaker Model For Vision-and-Language Navigation (Dou & Peng, NAACL 2022)

ACL

• Zi-Yi Dou and Nanyun Peng. 2022. FOAM: A Follower-aware Speaker Model For Vision-and-Language Navigation. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 4332–4340, Seattle, United States. Association for Computational Linguistics.