<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="dobrzeniecka-urbaniak-2024-bayesian">
    <titleInfo>
        <title>A Bayesian Approach to Uncertainty in Word Embedding Bias Estimation</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Alicja</namePart>
        <namePart type="family">Dobrzeniecka</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Rafal</namePart>
        <namePart type="family">Urbaniak</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2024-06</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <genre authority="bibutilsgt">journal article</genre>
    <relatedItem type="host">
        <titleInfo>
            <title>Computational Linguistics</title>
        </titleInfo>
        <originInfo>
            <issuance>continuing</issuance>
            <publisher>MIT Press</publisher>
            <place>
                <placeTerm type="text">Cambridge, MA</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">periodical</genre>
        <genre authority="bibutilsgt">academic journal</genre>
    </relatedItem>
    <abstract>Multiple measures, such as WEAT or MAC, attempt to quantify the magnitude of bias present in word embeddings in terms of a single-number metric. However, such metrics and the related statistical significance calculations rely on treating pre-averaged data as individual data points and utilizing bootstrapping techniques with low sample sizes. We show that similar results can be easily obtained using such methods even if the data are generated by a null model lacking the intended bias. Consequently, we argue that this approach generates false confidence. To address this issue, we propose a Bayesian alternative: hierarchical Bayesian modeling, which enables a more uncertainty-sensitive inspection of bias in word embeddings at different levels of granularity. To showcase our method, we apply it to Religion, Gender, and Race word lists from the original research, together with our control neutral word lists. We deploy the method using Google, GloVe, and Reddit embeddings. Further, we utilize our approach to evaluate a debiasing technique applied to the Reddit word embedding. Our findings reveal a more complex landscape than suggested by the proponents of single-number metrics. The datasets and source code for the paper are publicly available.1</abstract>
    <identifier type="citekey">dobrzeniecka-urbaniak-2024-bayesian</identifier>
    <identifier type="doi">10.1162/coli_a_00507</identifier>
    <location>
        <url>https://aclanthology.org/2024.cl-2.4</url>
    </location>
    <part>
        <date>2024-06</date>
        <detail type="volume"><number>50</number></detail>
        <detail type="issue"><number>2</number></detail>
        <extent unit="page">
            <start>563</start>
            <end>617</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Journal Article
%T A Bayesian Approach to Uncertainty in Word Embedding Bias Estimation
%A Dobrzeniecka, Alicja
%A Urbaniak, Rafal
%J Computational Linguistics
%D 2024
%8 June
%V 50
%N 2
%I MIT Press
%C Cambridge, MA
%F dobrzeniecka-urbaniak-2024-bayesian
%X Multiple measures, such as WEAT or MAC, attempt to quantify the magnitude of bias present in word embeddings in terms of a single-number metric. However, such metrics and the related statistical significance calculations rely on treating pre-averaged data as individual data points and utilizing bootstrapping techniques with low sample sizes. We show that similar results can be easily obtained using such methods even if the data are generated by a null model lacking the intended bias. Consequently, we argue that this approach generates false confidence. To address this issue, we propose a Bayesian alternative: hierarchical Bayesian modeling, which enables a more uncertainty-sensitive inspection of bias in word embeddings at different levels of granularity. To showcase our method, we apply it to Religion, Gender, and Race word lists from the original research, together with our control neutral word lists. We deploy the method using Google, GloVe, and Reddit embeddings. Further, we utilize our approach to evaluate a debiasing technique applied to the Reddit word embedding. Our findings reveal a more complex landscape than suggested by the proponents of single-number metrics. The datasets and source code for the paper are publicly available.1
%R 10.1162/coli_a_00507
%U https://aclanthology.org/2024.cl-2.4
%U https://doi.org/10.1162/coli_a_00507
%P 563-617

Markdown (Informal)

[A Bayesian Approach to Uncertainty in Word Embedding Bias Estimation](https://aclanthology.org/2024.cl-2.4) (Dobrzeniecka & Urbaniak, CL 2024)

• A Bayesian Approach to Uncertainty in Word Embedding Bias Estimation (Dobrzeniecka & Urbaniak, CL 2024)

ACL

• Alicja Dobrzeniecka and Rafal Urbaniak. 2024. A Bayesian Approach to Uncertainty in Word Embedding Bias Estimation. Computational Linguistics, 50(2):563–617.