<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="rotman-etal-2021-model">
    <titleInfo>
        <title>Model Compression for Domain Adaptation through Causal Effect Estimation</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Guy</namePart>
        <namePart type="family">Rotman</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Amir</namePart>
        <namePart type="family">Feder</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Roi</namePart>
        <namePart type="family">Reichart</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2021</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <genre authority="bibutilsgt">journal article</genre>
    <relatedItem type="host">
        <titleInfo>
            <title>Transactions of the Association for Computational Linguistics</title>
        </titleInfo>
        <originInfo>
            <issuance>continuing</issuance>
            <publisher>MIT Press</publisher>
            <place>
                <placeTerm type="text">Cambridge, MA</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">periodical</genre>
        <genre authority="bibutilsgt">academic journal</genre>
    </relatedItem>
    <abstract>Recent improvements in the predictive quality of natural language processing systems are often dependent on a substantial increase in the number of model parameters. This has led to various attempts of compressing such models, but existing methods have not considered the differences in the predictive power of various model components or in the generalizability of the compressed models. To understand the connection between model compression and out-of-distribution generalization, we define the task of compressing language representation models such that they perform best in a domain adaptation setting. We choose to address this problem from a causal perspective, attempting to estimate the average treatment effect (ATE) of a model component, such as a single layer, on the model’s predictions. Our proposed ATE-guided Model Compression scheme (AMoC), generates many model candidates, differing by the model components that were removed. Then, we select the best candidate through a stepwise regression model that utilizes the ATE to predict the expected performance on the target domain. AMoC outperforms strong baselines on dozens of domain pairs across three text classification and sequence tagging tasks.1</abstract>
    <identifier type="citekey">rotman-etal-2021-model</identifier>
    <identifier type="doi">10.1162/tacl_a_00431</identifier>
    <location>
        <url>https://aclanthology.org/2021.tacl-1.80</url>
    </location>
    <part>
        <date>2021</date>
        <detail type="volume"><number>9</number></detail>
        <extent unit="page">
            <start>1355</start>
            <end>1373</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Journal Article
%T Model Compression for Domain Adaptation through Causal Effect Estimation
%A Rotman, Guy
%A Feder, Amir
%A Reichart, Roi
%J Transactions of the Association for Computational Linguistics
%D 2021
%V 9
%I MIT Press
%C Cambridge, MA
%F rotman-etal-2021-model
%X Recent improvements in the predictive quality of natural language processing systems are often dependent on a substantial increase in the number of model parameters. This has led to various attempts of compressing such models, but existing methods have not considered the differences in the predictive power of various model components or in the generalizability of the compressed models. To understand the connection between model compression and out-of-distribution generalization, we define the task of compressing language representation models such that they perform best in a domain adaptation setting. We choose to address this problem from a causal perspective, attempting to estimate the average treatment effect (ATE) of a model component, such as a single layer, on the model’s predictions. Our proposed ATE-guided Model Compression scheme (AMoC), generates many model candidates, differing by the model components that were removed. Then, we select the best candidate through a stepwise regression model that utilizes the ATE to predict the expected performance on the target domain. AMoC outperforms strong baselines on dozens of domain pairs across three text classification and sequence tagging tasks.1
%R 10.1162/tacl_a_00431
%U https://aclanthology.org/2021.tacl-1.80
%U https://doi.org/10.1162/tacl_a_00431
%P 1355-1373

Markdown (Informal)

[Model Compression for Domain Adaptation through Causal Effect Estimation](https://aclanthology.org/2021.tacl-1.80) (Rotman et al., TACL 2021)

• Model Compression for Domain Adaptation through Causal Effect Estimation (Rotman et al., TACL 2021)

ACL

• Guy Rotman, Amir Feder, and Roi Reichart. 2021. Model Compression for Domain Adaptation through Causal Effect Estimation. Transactions of the Association for Computational Linguistics, 9:1355–1373.