<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="yu-rieser-2023-adversarial">
    <titleInfo>
        <title>Adversarial Textual Robustness on Visual Dialog</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Lu</namePart>
        <namePart type="family">Yu</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Verena</namePart>
        <namePart type="family">Rieser</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2023-07</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Findings of the Association for Computational Linguistics: ACL 2023</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Anna</namePart>
            <namePart type="family">Rogers</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Jordan</namePart>
            <namePart type="family">Boyd-Graber</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Naoaki</namePart>
            <namePart type="family">Okazaki</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Toronto, Canada</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>Adversarial robustness evaluates the worst-case performance scenario of a machine learning model to ensure its safety and reliability. For example, cases where the user input contains a minimal change, e.g. a synonym, which causes the previously correct model to return a wrong answer. Using this scenario, this study is the first to investigate the robustness of visually grounded dialog models towards textual attacks. We first aim to understand how multimodal input components contribute to model robustness. Our results show that models which encode dialog history are more robust by providing redundant information. This is in contrast to prior work which finds that dialog history is negligible for model performance on this task. We also evaluate how to generate adversarial test examples which successfully fool the model but remain undetected by the user/software designer. Our analysis shows that the textual, as well as the visual context are important to generate plausible attacks.</abstract>
    <identifier type="citekey">yu-rieser-2023-adversarial</identifier>
    <identifier type="doi">10.18653/v1/2023.findings-acl.212</identifier>
    <location>
        <url>https://aclanthology.org/2023.findings-acl.212</url>
    </location>
    <part>
        <date>2023-07</date>
        <extent unit="page">
            <start>3422</start>
            <end>3438</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Adversarial Textual Robustness on Visual Dialog
%A Yu, Lu
%A Rieser, Verena
%Y Rogers, Anna
%Y Boyd-Graber, Jordan
%Y Okazaki, Naoaki
%S Findings of the Association for Computational Linguistics: ACL 2023
%D 2023
%8 July
%I Association for Computational Linguistics
%C Toronto, Canada
%F yu-rieser-2023-adversarial
%X Adversarial robustness evaluates the worst-case performance scenario of a machine learning model to ensure its safety and reliability. For example, cases where the user input contains a minimal change, e.g. a synonym, which causes the previously correct model to return a wrong answer. Using this scenario, this study is the first to investigate the robustness of visually grounded dialog models towards textual attacks. We first aim to understand how multimodal input components contribute to model robustness. Our results show that models which encode dialog history are more robust by providing redundant information. This is in contrast to prior work which finds that dialog history is negligible for model performance on this task. We also evaluate how to generate adversarial test examples which successfully fool the model but remain undetected by the user/software designer. Our analysis shows that the textual, as well as the visual context are important to generate plausible attacks.
%R 10.18653/v1/2023.findings-acl.212
%U https://aclanthology.org/2023.findings-acl.212
%U https://doi.org/10.18653/v1/2023.findings-acl.212
%P 3422-3438

Markdown (Informal)

[Adversarial Textual Robustness on Visual Dialog](https://aclanthology.org/2023.findings-acl.212) (Yu & Rieser, Findings 2023)

• Adversarial Textual Robustness on Visual Dialog (Yu & Rieser, Findings 2023)

ACL

• Lu Yu and Verena Rieser. 2023. Adversarial Textual Robustness on Visual Dialog. In Findings of the Association for Computational Linguistics: ACL 2023, pages 3422–3438, Toronto, Canada. Association for Computational Linguistics.