<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="wang-etal-2020-learning-efficient">
    <titleInfo>
        <title>Learning Efficient Dialogue Policy from Demonstrations through Shaping</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Huimin</namePart>
        <namePart type="family">Wang</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Baolin</namePart>
        <namePart type="family">Peng</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Kam-Fai</namePart>
        <namePart type="family">Wong</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2020-07</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Dan</namePart>
            <namePart type="family">Jurafsky</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Joyce</namePart>
            <namePart type="family">Chai</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Natalie</namePart>
            <namePart type="family">Schluter</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Joel</namePart>
            <namePart type="family">Tetreault</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Online</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>Training a task-oriented dialogue agent with reinforcement learning is prohibitively expensive since it requires a large volume of interactions with users. Human demonstrations can be used to accelerate learning progress. However, how to effectively leverage demonstrations to learn dialogue policy remains less explored. In this paper, we present S\²Agent that efficiently learns dialogue policy from demonstrations through policy shaping and reward shaping. We use an imitation model to distill knowledge from demonstrations, based on which policy shaping estimates feedback on how the agent should act in policy space. Reward shaping is then incorporated to bonus state-actions similar to demonstrations explicitly in value space encouraging better exploration. The effectiveness of the proposed S\²Agentt is demonstrated in three dialogue domains and a challenging domain adaptation task with both user simulator evaluation and human evaluation.</abstract>
    <identifier type="citekey">wang-etal-2020-learning-efficient</identifier>
    <identifier type="doi">10.18653/v1/2020.acl-main.566</identifier>
    <location>
        <url>https://aclanthology.org/2020.acl-main.566</url>
    </location>
    <part>
        <date>2020-07</date>
        <extent unit="page">
            <start>6355</start>
            <end>6365</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Learning Efficient Dialogue Policy from Demonstrations through Shaping
%A Wang, Huimin
%A Peng, Baolin
%A Wong, Kam-Fai
%Y Jurafsky, Dan
%Y Chai, Joyce
%Y Schluter, Natalie
%Y Tetreault, Joel
%S Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics
%D 2020
%8 July
%I Association for Computational Linguistics
%C Online
%F wang-etal-2020-learning-efficient
%X Training a task-oriented dialogue agent with reinforcement learning is prohibitively expensive since it requires a large volume of interactions with users. Human demonstrations can be used to accelerate learning progress. However, how to effectively leverage demonstrations to learn dialogue policy remains less explored. In this paper, we present S\²Agent that efficiently learns dialogue policy from demonstrations through policy shaping and reward shaping. We use an imitation model to distill knowledge from demonstrations, based on which policy shaping estimates feedback on how the agent should act in policy space. Reward shaping is then incorporated to bonus state-actions similar to demonstrations explicitly in value space encouraging better exploration. The effectiveness of the proposed S\²Agentt is demonstrated in three dialogue domains and a challenging domain adaptation task with both user simulator evaluation and human evaluation.
%R 10.18653/v1/2020.acl-main.566
%U https://aclanthology.org/2020.acl-main.566
%U https://doi.org/10.18653/v1/2020.acl-main.566
%P 6355-6365

Markdown (Informal)

[Learning Efficient Dialogue Policy from Demonstrations through Shaping](https://aclanthology.org/2020.acl-main.566) (Wang et al., ACL 2020)

• Learning Efficient Dialogue Policy from Demonstrations through Shaping (Wang et al., ACL 2020)

ACL

• Huimin Wang, Baolin Peng, and Kam-Fai Wong. 2020. Learning Efficient Dialogue Policy from Demonstrations through Shaping. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 6355–6365, Online. Association for Computational Linguistics.