<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="senge-etal-2022-one">
    <titleInfo>
        <title>One size does not fit all: Investigating strategies for differentially-private learning across NLP tasks</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Manuel</namePart>
        <namePart type="family">Senge</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Timour</namePart>
        <namePart type="family">Igamberdiev</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Ivan</namePart>
        <namePart type="family">Habernal</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2022-12</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Yoav</namePart>
            <namePart type="family">Goldberg</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Zornitsa</namePart>
            <namePart type="family">Kozareva</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Yue</namePart>
            <namePart type="family">Zhang</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Abu Dhabi, United Arab Emirates</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>Preserving privacy in contemporary NLP models allows us to work with sensitive data, but unfortunately comes at a price. We know that stricter privacy guarantees in differentially-private stochastic gradient descent (DP-SGD) generally degrade model performance. However, previous research on the efficiency of DP-SGD in NLP is inconclusive or even counter-intuitive. In this short paper, we provide an extensive analysis of different privacy preserving strategies on seven downstream datasets in five different ‘typical’ NLP tasks with varying complexity using modern neural models based on BERT and XtremeDistil architectures. We show that unlike standard non-private approaches to solving NLP tasks, where bigger is usually better, privacy-preserving strategies do not exhibit a winning pattern, and each task and privacy regime requires a special treatment to achieve adequate performance.</abstract>
    <identifier type="citekey">senge-etal-2022-one</identifier>
    <identifier type="doi">10.18653/v1/2022.emnlp-main.496</identifier>
    <location>
        <url>https://aclanthology.org/2022.emnlp-main.496</url>
    </location>
    <part>
        <date>2022-12</date>
        <extent unit="page">
            <start>7340</start>
            <end>7353</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T One size does not fit all: Investigating strategies for differentially-private learning across NLP tasks
%A Senge, Manuel
%A Igamberdiev, Timour
%A Habernal, Ivan
%Y Goldberg, Yoav
%Y Kozareva, Zornitsa
%Y Zhang, Yue
%S Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing
%D 2022
%8 December
%I Association for Computational Linguistics
%C Abu Dhabi, United Arab Emirates
%F senge-etal-2022-one
%X Preserving privacy in contemporary NLP models allows us to work with sensitive data, but unfortunately comes at a price. We know that stricter privacy guarantees in differentially-private stochastic gradient descent (DP-SGD) generally degrade model performance. However, previous research on the efficiency of DP-SGD in NLP is inconclusive or even counter-intuitive. In this short paper, we provide an extensive analysis of different privacy preserving strategies on seven downstream datasets in five different ‘typical’ NLP tasks with varying complexity using modern neural models based on BERT and XtremeDistil architectures. We show that unlike standard non-private approaches to solving NLP tasks, where bigger is usually better, privacy-preserving strategies do not exhibit a winning pattern, and each task and privacy regime requires a special treatment to achieve adequate performance.
%R 10.18653/v1/2022.emnlp-main.496
%U https://aclanthology.org/2022.emnlp-main.496
%U https://doi.org/10.18653/v1/2022.emnlp-main.496
%P 7340-7353

Markdown (Informal)

[One size does not fit all: Investigating strategies for differentially-private learning across NLP tasks](https://aclanthology.org/2022.emnlp-main.496) (Senge et al., EMNLP 2022)

• One size does not fit all: Investigating strategies for differentially-private learning across NLP tasks (Senge et al., EMNLP 2022)

ACL

• Manuel Senge, Timour Igamberdiev, and Ivan Habernal. 2022. One size does not fit all: Investigating strategies for differentially-private learning across NLP tasks. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 7340–7353, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics.