<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="dayanik-etal-2022-analysis">
    <titleInfo>
        <title>Bias Identification and Attribution in NLP Models With Regression and Effect Sizes</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Erenay</namePart>
        <namePart type="family">Dayanik</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Ngoc</namePart>
        <namePart type="given">Thang</namePart>
        <namePart type="family">Vu</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Sebastian</namePart>
        <namePart type="family">Padó</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2022</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Northern European Journal of Language Technology, Volume 8</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Leon</namePart>
            <namePart type="family">Derczynski</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Northern European Association of Language Technology</publisher>
            <place>
                <placeTerm type="text">Copenhagen, Denmark</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>In recent years, there has been an increasing awareness that many NLP systems incorporate biases of various types (e.g., regarding gender or race) which can have significant negative consequences. At the same time, the techniques used to statistically analyze such biases are still relatively simple. Typically, studies test for the presence of a significant difference between two levels of a single bias variable (e.g., male vs. female) without attention to potential confounders, and do not quantify the importance of the bias variable. This article proposes to analyze bias in the output of NLP systems using multivariate regression models. They provide a robust and more informative alternative which (a) generalizes to multiple bias variables, (b) can take covariates into account, (c) can be combined with measures of effect size to quantify the size of bias. Jointly, these effects contribute to a more robust statistical analysis of bias that can be used to diagnose system behavior and extract informative examples. We demonstrate the benefits of our method by analyzing a range of current NLP models on one regression and one classification tasks (emotion intensity prediction and coreference resolution, respectively).</abstract>
    <identifier type="citekey">dayanik-etal-2022-analysis</identifier>
    <identifier type="doi">https://doi.org/10.3384/nejlt.2000-1533.2022.3505</identifier>
    <location>
        <url>https://aclanthology.org/2022.nejlt-1.4</url>
    </location>
    <part>
        <date>2022</date>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Bias Identification and Attribution in NLP Models With Regression and Effect Sizes
%A Dayanik, Erenay
%A Vu, Ngoc Thang
%A Padó, Sebastian
%Y Derczynski, Leon
%S Northern European Journal of Language Technology, Volume 8
%D 2022
%I Northern European Association of Language Technology
%C Copenhagen, Denmark
%F dayanik-etal-2022-analysis
%X In recent years, there has been an increasing awareness that many NLP systems incorporate biases of various types (e.g., regarding gender or race) which can have significant negative consequences. At the same time, the techniques used to statistically analyze such biases are still relatively simple. Typically, studies test for the presence of a significant difference between two levels of a single bias variable (e.g., male vs. female) without attention to potential confounders, and do not quantify the importance of the bias variable. This article proposes to analyze bias in the output of NLP systems using multivariate regression models. They provide a robust and more informative alternative which (a) generalizes to multiple bias variables, (b) can take covariates into account, (c) can be combined with measures of effect size to quantify the size of bias. Jointly, these effects contribute to a more robust statistical analysis of bias that can be used to diagnose system behavior and extract informative examples. We demonstrate the benefits of our method by analyzing a range of current NLP models on one regression and one classification tasks (emotion intensity prediction and coreference resolution, respectively).
%R https://doi.org/10.3384/nejlt.2000-1533.2022.3505
%U https://aclanthology.org/2022.nejlt-1.4
%U https://doi.org/https://doi.org/10.3384/nejlt.2000-1533.2022.3505

Markdown (Informal)

[Bias Identification and Attribution in NLP Models With Regression and Effect Sizes](https://aclanthology.org/2022.nejlt-1.4) (Dayanik et al., NEJLT 2022)

• Bias Identification and Attribution in NLP Models With Regression and Effect Sizes (Dayanik et al., NEJLT 2022)

ACL

• Erenay Dayanik, Ngoc Thang Vu, and Sebastian Padó. 2022. Bias Identification and Attribution in NLP Models With Regression and Effect Sizes. In Northern European Journal of Language Technology, Volume 8, Copenhagen, Denmark. Northern European Association of Language Technology.