<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="ivison-peters-2022-hyperdecoders">
    <titleInfo>
        <title>Hyperdecoders: Instance-specific decoders for multi-task NLP</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Hamish</namePart>
        <namePart type="family">Ivison</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Matthew</namePart>
        <namePart type="family">Peters</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2022-12</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Findings of the Association for Computational Linguistics: EMNLP 2022</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Yoav</namePart>
            <namePart type="family">Goldberg</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Zornitsa</namePart>
            <namePart type="family">Kozareva</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Yue</namePart>
            <namePart type="family">Zhang</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Abu Dhabi, United Arab Emirates</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>We investigate input-conditioned hypernetworks for multi-tasking in NLP, generating parameter-efficient adaptations for a decoder using a hypernetwork conditioned on the output of an encoder. This approach produces a unique decoder adaptation for every input instance, allowing the network a larger degree of flexibility than prior work that only produces one decoder adaptation per task. We apply our method to sequence classification tasks, extractive QA, and summarisation and find that it surpasses previous parameter efficient fine-tuning methods and often outperforms fully finetuning the underlying model. An analysis of the embeddings used by our hypernetwork shows that they are sensitive to output label and type, suggesting that our approach better maps from encoder representations to output labels. Our code is publicly available at https://github.com/allenai/hyperdecoders.</abstract>
    <identifier type="citekey">ivison-peters-2022-hyperdecoders</identifier>
    <identifier type="doi">10.18653/v1/2022.findings-emnlp.124</identifier>
    <location>
        <url>https://aclanthology.org/2022.findings-emnlp.124</url>
    </location>
    <part>
        <date>2022-12</date>
        <extent unit="page">
            <start>1715</start>
            <end>1730</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Hyperdecoders: Instance-specific decoders for multi-task NLP
%A Ivison, Hamish
%A Peters, Matthew
%Y Goldberg, Yoav
%Y Kozareva, Zornitsa
%Y Zhang, Yue
%S Findings of the Association for Computational Linguistics: EMNLP 2022
%D 2022
%8 December
%I Association for Computational Linguistics
%C Abu Dhabi, United Arab Emirates
%F ivison-peters-2022-hyperdecoders
%X We investigate input-conditioned hypernetworks for multi-tasking in NLP, generating parameter-efficient adaptations for a decoder using a hypernetwork conditioned on the output of an encoder. This approach produces a unique decoder adaptation for every input instance, allowing the network a larger degree of flexibility than prior work that only produces one decoder adaptation per task. We apply our method to sequence classification tasks, extractive QA, and summarisation and find that it surpasses previous parameter efficient fine-tuning methods and often outperforms fully finetuning the underlying model. An analysis of the embeddings used by our hypernetwork shows that they are sensitive to output label and type, suggesting that our approach better maps from encoder representations to output labels. Our code is publicly available at https://github.com/allenai/hyperdecoders.
%R 10.18653/v1/2022.findings-emnlp.124
%U https://aclanthology.org/2022.findings-emnlp.124
%U https://doi.org/10.18653/v1/2022.findings-emnlp.124
%P 1715-1730

Markdown (Informal)

[Hyperdecoders: Instance-specific decoders for multi-task NLP](https://aclanthology.org/2022.findings-emnlp.124) (Ivison & Peters, Findings 2022)

• Hyperdecoders: Instance-specific decoders for multi-task NLP (Ivison & Peters, Findings 2022)

ACL

• Hamish Ivison and Matthew Peters. 2022. Hyperdecoders: Instance-specific decoders for multi-task NLP. In Findings of the Association for Computational Linguistics: EMNLP 2022, pages 1715–1730, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics.