<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="agarwal-etal-2023-peftdebias">
    <titleInfo>
        <title>PEFTDebias : Capturing debiasing information using PEFTs</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Sumit</namePart>
        <namePart type="family">Agarwal</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Aditya</namePart>
        <namePart type="family">Veerubhotla</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Srijan</namePart>
        <namePart type="family">Bansal</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2023-12</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Houda</namePart>
            <namePart type="family">Bouamor</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Juan</namePart>
            <namePart type="family">Pino</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Kalika</namePart>
            <namePart type="family">Bali</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Singapore</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>The increasing use of foundation models highlights the urgent need to address and eliminate implicit biases present in them that arise during pretraining. In this paper, we introduce PEFTDebias, a novel approach that employs parameter-efficient fine-tuning (PEFT) to mitigate the biases within foundation models. PEFTDebias consists of two main phases: an upstream phase for acquiring debiasing parameters along a specific bias axis, and a downstream phase where these parameters are incorporated into the model and frozen during the fine-tuning process. By evaluating on four datasets across two bias axes namely gender and race, we find that downstream biases can be effectively reduced with PEFTs. In addition, we show that these parameters possess axis-specific debiasing characteristics, enabling their effective transferability in mitigating biases in various downstream tasks.</abstract>
    <identifier type="citekey">agarwal-etal-2023-peftdebias</identifier>
    <identifier type="doi">10.18653/v1/2023.emnlp-main.122</identifier>
    <location>
        <url>https://aclanthology.org/2023.emnlp-main.122</url>
    </location>
    <part>
        <date>2023-12</date>
        <extent unit="page">
            <start>1992</start>
            <end>2000</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T PEFTDebias : Capturing debiasing information using PEFTs
%A Agarwal, Sumit
%A Veerubhotla, Aditya
%A Bansal, Srijan
%Y Bouamor, Houda
%Y Pino, Juan
%Y Bali, Kalika
%S Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing
%D 2023
%8 December
%I Association for Computational Linguistics
%C Singapore
%F agarwal-etal-2023-peftdebias
%X The increasing use of foundation models highlights the urgent need to address and eliminate implicit biases present in them that arise during pretraining. In this paper, we introduce PEFTDebias, a novel approach that employs parameter-efficient fine-tuning (PEFT) to mitigate the biases within foundation models. PEFTDebias consists of two main phases: an upstream phase for acquiring debiasing parameters along a specific bias axis, and a downstream phase where these parameters are incorporated into the model and frozen during the fine-tuning process. By evaluating on four datasets across two bias axes namely gender and race, we find that downstream biases can be effectively reduced with PEFTs. In addition, we show that these parameters possess axis-specific debiasing characteristics, enabling their effective transferability in mitigating biases in various downstream tasks.
%R 10.18653/v1/2023.emnlp-main.122
%U https://aclanthology.org/2023.emnlp-main.122
%U https://doi.org/10.18653/v1/2023.emnlp-main.122
%P 1992-2000

Markdown (Informal)

[PEFTDebias : Capturing debiasing information using PEFTs](https://aclanthology.org/2023.emnlp-main.122) (Agarwal et al., EMNLP 2023)

• PEFTDebias : Capturing debiasing information using PEFTs (Agarwal et al., EMNLP 2023)

ACL

• Sumit Agarwal, Aditya Veerubhotla, and Srijan Bansal. 2023. PEFTDebias : Capturing debiasing information using PEFTs. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 1992–2000, Singapore. Association for Computational Linguistics.