<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="gheini-etal-2023-know">
    <titleInfo>
        <title>Know Where You’re Going: Meta-Learning for Parameter-Efficient Fine-Tuning</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Mozhdeh</namePart>
        <namePart type="family">Gheini</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Xuezhe</namePart>
        <namePart type="family">Ma</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Jonathan</namePart>
        <namePart type="family">May</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2023-07</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Findings of the Association for Computational Linguistics: ACL 2023</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Anna</namePart>
            <namePart type="family">Rogers</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Jordan</namePart>
            <namePart type="family">Boyd-Graber</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Naoaki</namePart>
            <namePart type="family">Okazaki</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Toronto, Canada</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>A recent family of techniques, dubbed lightweight fine-tuning methods, facilitates parameter-efficient transfer by updating only a small set of additional parameters while keeping the parameters of the original model frozen. While proven to be an effective approach, there are no existing studies on if and how such knowledge of the downstream fine-tuning approach calls for complementary measures after pre-training and before fine-tuning. In this work, we show that taking the ultimate choice of fine-tuning into consideration boosts the performance of parameter-efficient fine-tuning. By relying on optimization-based meta-learning using MAML with certain modifications for our distinct purpose, we prime the pre-trained model specifically for parameter-efficient fine-tuning, resulting in gains of up to 4.96 points on cross-lingual NER fine-tuning. Our ablation settings and analyses further reveal that the specific approach we take to meta-learning is crucial for the attained gains.</abstract>
    <identifier type="citekey">gheini-etal-2023-know</identifier>
    <identifier type="doi">10.18653/v1/2023.findings-acl.737</identifier>
    <location>
        <url>https://aclanthology.org/2023.findings-acl.737</url>
    </location>
    <part>
        <date>2023-07</date>
        <extent unit="page">
            <start>11602</start>
            <end>11612</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Know Where You’re Going: Meta-Learning for Parameter-Efficient Fine-Tuning
%A Gheini, Mozhdeh
%A Ma, Xuezhe
%A May, Jonathan
%Y Rogers, Anna
%Y Boyd-Graber, Jordan
%Y Okazaki, Naoaki
%S Findings of the Association for Computational Linguistics: ACL 2023
%D 2023
%8 July
%I Association for Computational Linguistics
%C Toronto, Canada
%F gheini-etal-2023-know
%X A recent family of techniques, dubbed lightweight fine-tuning methods, facilitates parameter-efficient transfer by updating only a small set of additional parameters while keeping the parameters of the original model frozen. While proven to be an effective approach, there are no existing studies on if and how such knowledge of the downstream fine-tuning approach calls for complementary measures after pre-training and before fine-tuning. In this work, we show that taking the ultimate choice of fine-tuning into consideration boosts the performance of parameter-efficient fine-tuning. By relying on optimization-based meta-learning using MAML with certain modifications for our distinct purpose, we prime the pre-trained model specifically for parameter-efficient fine-tuning, resulting in gains of up to 4.96 points on cross-lingual NER fine-tuning. Our ablation settings and analyses further reveal that the specific approach we take to meta-learning is crucial for the attained gains.
%R 10.18653/v1/2023.findings-acl.737
%U https://aclanthology.org/2023.findings-acl.737
%U https://doi.org/10.18653/v1/2023.findings-acl.737
%P 11602-11612

Markdown (Informal)

[Know Where You’re Going: Meta-Learning for Parameter-Efficient Fine-Tuning](https://aclanthology.org/2023.findings-acl.737) (Gheini et al., Findings 2023)

• Know Where You’re Going: Meta-Learning for Parameter-Efficient Fine-Tuning (Gheini et al., Findings 2023)

ACL

• Mozhdeh Gheini, Xuezhe Ma, and Jonathan May. 2023. Know Where You’re Going: Meta-Learning for Parameter-Efficient Fine-Tuning. In Findings of the Association for Computational Linguistics: ACL 2023, pages 11602–11612, Toronto, Canada. Association for Computational Linguistics.