<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="pugantsov-mccreadie-2023-divergence">
    <titleInfo>
        <title>Divergence-Based Domain Transferability for Zero-Shot Classification</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Alexander</namePart>
        <namePart type="family">Pugantsov</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Richard</namePart>
        <namePart type="family">McCreadie</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2023-05</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Findings of the Association for Computational Linguistics: EACL 2023</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Andreas</namePart>
            <namePart type="family">Vlachos</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Isabelle</namePart>
            <namePart type="family">Augenstein</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Dubrovnik, Croatia</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>Transferring learned patterns from pretrained neural language models has been shown to significantly improve effectiveness across a variety of language-based tasks, meanwhile further tuning on intermediate tasks has been demonstrated to provide additional performance benefits, provided the intermediate task is sufficiently related to the target task. However, how to identify related tasks is an open problem, and brute-force searching effective task combinations is prohibitively expensive. Hence, the question arises, are we able to improve the effectiveness and efficiency of tasks with no training examples through selective fine-tuning? In this paper, we explore statistical measures that approximate the divergence between domain representations as a means to estimate whether tuning using one task pair will exhibit performance benefits over tuning another. This estimation can then be used to reduce the number of task pairs that need to be tested by eliminating pairs that are unlikely to provide benefits. Through experimentation over 58 tasks and over 6,600 task pair combinations, we demonstrate that statistical measures can distinguish effective task pairs, and the resulting estimates can reduce end-to-end runtime by up to 40%.</abstract>
    <identifier type="citekey">pugantsov-mccreadie-2023-divergence</identifier>
    <identifier type="doi">10.18653/v1/2023.findings-eacl.122</identifier>
    <location>
        <url>https://aclanthology.org/2023.findings-eacl.122</url>
    </location>
    <part>
        <date>2023-05</date>
        <extent unit="page">
            <start>1649</start>
            <end>1654</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Divergence-Based Domain Transferability for Zero-Shot Classification
%A Pugantsov, Alexander
%A McCreadie, Richard
%Y Vlachos, Andreas
%Y Augenstein, Isabelle
%S Findings of the Association for Computational Linguistics: EACL 2023
%D 2023
%8 May
%I Association for Computational Linguistics
%C Dubrovnik, Croatia
%F pugantsov-mccreadie-2023-divergence
%X Transferring learned patterns from pretrained neural language models has been shown to significantly improve effectiveness across a variety of language-based tasks, meanwhile further tuning on intermediate tasks has been demonstrated to provide additional performance benefits, provided the intermediate task is sufficiently related to the target task. However, how to identify related tasks is an open problem, and brute-force searching effective task combinations is prohibitively expensive. Hence, the question arises, are we able to improve the effectiveness and efficiency of tasks with no training examples through selective fine-tuning? In this paper, we explore statistical measures that approximate the divergence between domain representations as a means to estimate whether tuning using one task pair will exhibit performance benefits over tuning another. This estimation can then be used to reduce the number of task pairs that need to be tested by eliminating pairs that are unlikely to provide benefits. Through experimentation over 58 tasks and over 6,600 task pair combinations, we demonstrate that statistical measures can distinguish effective task pairs, and the resulting estimates can reduce end-to-end runtime by up to 40%.
%R 10.18653/v1/2023.findings-eacl.122
%U https://aclanthology.org/2023.findings-eacl.122
%U https://doi.org/10.18653/v1/2023.findings-eacl.122
%P 1649-1654

Markdown (Informal)

[Divergence-Based Domain Transferability for Zero-Shot Classification](https://aclanthology.org/2023.findings-eacl.122) (Pugantsov & McCreadie, Findings 2023)

• Divergence-Based Domain Transferability for Zero-Shot Classification (Pugantsov & McCreadie, Findings 2023)

ACL

• Alexander Pugantsov and Richard McCreadie. 2023. Divergence-Based Domain Transferability for Zero-Shot Classification. In Findings of the Association for Computational Linguistics: EACL 2023, pages 1649–1654, Dubrovnik, Croatia. Association for Computational Linguistics.