<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="thakkar-etal-2021-understanding">
    <titleInfo>
        <title>Understanding Unintended Memorization in Language Models Under Federated Learning</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Om</namePart>
        <namePart type="given">Dipakbhai</namePart>
        <namePart type="family">Thakkar</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Swaroop</namePart>
        <namePart type="family">Ramaswamy</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Rajiv</namePart>
        <namePart type="family">Mathews</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Francoise</namePart>
        <namePart type="family">Beaufays</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2021-06</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the Third Workshop on Privacy in Natural Language Processing</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Oluwaseyi</namePart>
            <namePart type="family">Feyisetan</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Sepideh</namePart>
            <namePart type="family">Ghanavati</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Shervin</namePart>
            <namePart type="family">Malmasi</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Patricia</namePart>
            <namePart type="family">Thaine</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Online</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>Recent works have shown that language models (LMs), e.g., for next word prediction (NWP), have a tendency to memorize rare or unique sequences in the training data. Since useful LMs are often trained on sensitive data, it is critical to identify and mitigate such unintended memorization. Federated Learning (FL) has emerged as a novel framework for large-scale distributed learning tasks. It differs in many aspects from the well-studied central learning setting where all the data is stored at the central server, and minibatch stochastic gradient descent is used to conduct training. This work is motivated by our observation that NWP models trained under FL exhibited remarkably less propensity to such memorization compared to the central learning setting. Thus, we initiate a formal study to understand the effect of different components of FL on unintended memorization in trained NWP models. Our results show that several differing components of FL play an important role in reducing unintended memorization. First, we discover that the clustering of data according to users—which happens by design in FL—has the most significant effect in reducing such memorization. Using the Federated Averaging optimizer with larger effective minibatch sizes for training causes a further reduction. We also demonstrate that training in FL with a user-level differential privacy guarantee results in models that can provide high utility while being resilient to memorizing out-of-distribution phrases with thousands of insertions across over a hundred users in the training set.</abstract>
    <identifier type="citekey">thakkar-etal-2021-understanding</identifier>
    <identifier type="doi">10.18653/v1/2021.privatenlp-1.1</identifier>
    <location>
        <url>https://aclanthology.org/2021.privatenlp-1.1</url>
    </location>
    <part>
        <date>2021-06</date>
        <extent unit="page">
            <start>1</start>
            <end>10</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Understanding Unintended Memorization in Language Models Under Federated Learning
%A Thakkar, Om Dipakbhai
%A Ramaswamy, Swaroop
%A Mathews, Rajiv
%A Beaufays, Francoise
%Y Feyisetan, Oluwaseyi
%Y Ghanavati, Sepideh
%Y Malmasi, Shervin
%Y Thaine, Patricia
%S Proceedings of the Third Workshop on Privacy in Natural Language Processing
%D 2021
%8 June
%I Association for Computational Linguistics
%C Online
%F thakkar-etal-2021-understanding
%X Recent works have shown that language models (LMs), e.g., for next word prediction (NWP), have a tendency to memorize rare or unique sequences in the training data. Since useful LMs are often trained on sensitive data, it is critical to identify and mitigate such unintended memorization. Federated Learning (FL) has emerged as a novel framework for large-scale distributed learning tasks. It differs in many aspects from the well-studied central learning setting where all the data is stored at the central server, and minibatch stochastic gradient descent is used to conduct training. This work is motivated by our observation that NWP models trained under FL exhibited remarkably less propensity to such memorization compared to the central learning setting. Thus, we initiate a formal study to understand the effect of different components of FL on unintended memorization in trained NWP models. Our results show that several differing components of FL play an important role in reducing unintended memorization. First, we discover that the clustering of data according to users—which happens by design in FL—has the most significant effect in reducing such memorization. Using the Federated Averaging optimizer with larger effective minibatch sizes for training causes a further reduction. We also demonstrate that training in FL with a user-level differential privacy guarantee results in models that can provide high utility while being resilient to memorizing out-of-distribution phrases with thousands of insertions across over a hundred users in the training set.
%R 10.18653/v1/2021.privatenlp-1.1
%U https://aclanthology.org/2021.privatenlp-1.1
%U https://doi.org/10.18653/v1/2021.privatenlp-1.1
%P 1-10

Markdown (Informal)

[Understanding Unintended Memorization in Language Models Under Federated Learning](https://aclanthology.org/2021.privatenlp-1.1) (Thakkar et al., PrivateNLP 2021)

• Understanding Unintended Memorization in Language Models Under Federated Learning (Thakkar et al., PrivateNLP 2021)

ACL

• Om Dipakbhai Thakkar, Swaroop Ramaswamy, Rajiv Mathews, and Francoise Beaufays. 2021. Understanding Unintended Memorization in Language Models Under Federated Learning. In Proceedings of the Third Workshop on Privacy in Natural Language Processing, pages 1–10, Online. Association for Computational Linguistics.