<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="dong-etal-2023-promptattack">
    <titleInfo>
        <title>PromptAttack: Probing Dialogue State Trackers with Adversarial Prompts</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Xiangjue</namePart>
        <namePart type="family">Dong</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Yun</namePart>
        <namePart type="family">He</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Ziwei</namePart>
        <namePart type="family">Zhu</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">James</namePart>
        <namePart type="family">Caverlee</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2023-07</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Findings of the Association for Computational Linguistics: ACL 2023</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Anna</namePart>
            <namePart type="family">Rogers</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Jordan</namePart>
            <namePart type="family">Boyd-Graber</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Naoaki</namePart>
            <namePart type="family">Okazaki</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Toronto, Canada</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>A key component of modern conversational systems is the Dialogue State Tracker (or DST), which models a user’s goals and needs. Toward building more robust and reliable DSTs, we introduce a prompt-based learning approach to automatically generate effective adversarial examples to probe DST models. Two key characteristics of this approach are: (i) it only needs the output of the DST with no need for model parameters, and (ii) it can learn to generate natural language utterances that can target any DST. Through experiments over state-of-the-art DSTs, the proposed framework leads to the greatest reduction in accuracy and the best attack success rate while maintaining good fluency and a low perturbation ratio. We also show how much the generated adversarial examples can bolster a DST through adversarial training. These results indicate the strength of prompt-based attacks on DSTs and leave open avenues for continued refinement.</abstract>
    <identifier type="citekey">dong-etal-2023-promptattack</identifier>
    <identifier type="doi">10.18653/v1/2023.findings-acl.677</identifier>
    <location>
        <url>https://aclanthology.org/2023.findings-acl.677</url>
    </location>
    <part>
        <date>2023-07</date>
        <extent unit="page">
            <start>10651</start>
            <end>10666</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T PromptAttack: Probing Dialogue State Trackers with Adversarial Prompts
%A Dong, Xiangjue
%A He, Yun
%A Zhu, Ziwei
%A Caverlee, James
%Y Rogers, Anna
%Y Boyd-Graber, Jordan
%Y Okazaki, Naoaki
%S Findings of the Association for Computational Linguistics: ACL 2023
%D 2023
%8 July
%I Association for Computational Linguistics
%C Toronto, Canada
%F dong-etal-2023-promptattack
%X A key component of modern conversational systems is the Dialogue State Tracker (or DST), which models a user’s goals and needs. Toward building more robust and reliable DSTs, we introduce a prompt-based learning approach to automatically generate effective adversarial examples to probe DST models. Two key characteristics of this approach are: (i) it only needs the output of the DST with no need for model parameters, and (ii) it can learn to generate natural language utterances that can target any DST. Through experiments over state-of-the-art DSTs, the proposed framework leads to the greatest reduction in accuracy and the best attack success rate while maintaining good fluency and a low perturbation ratio. We also show how much the generated adversarial examples can bolster a DST through adversarial training. These results indicate the strength of prompt-based attacks on DSTs and leave open avenues for continued refinement.
%R 10.18653/v1/2023.findings-acl.677
%U https://aclanthology.org/2023.findings-acl.677
%U https://doi.org/10.18653/v1/2023.findings-acl.677
%P 10651-10666

Markdown (Informal)

[PromptAttack: Probing Dialogue State Trackers with Adversarial Prompts](https://aclanthology.org/2023.findings-acl.677) (Dong et al., Findings 2023)

• PromptAttack: Probing Dialogue State Trackers with Adversarial Prompts (Dong et al., Findings 2023)

ACL

• Xiangjue Dong, Yun He, Ziwei Zhu, and James Caverlee. 2023. PromptAttack: Probing Dialogue State Trackers with Adversarial Prompts. In Findings of the Association for Computational Linguistics: ACL 2023, pages 10651–10666, Toronto, Canada. Association for Computational Linguistics.