<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="arnett-etal-2024-bit">
    <titleInfo>
        <title>A Bit of a Problem: Measurement Disparities in Dataset Sizes across Languages</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Catherine</namePart>
        <namePart type="family">Arnett</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Tyler</namePart>
        <namePart type="given">A</namePart>
        <namePart type="family">Chang</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Benjamin</namePart>
        <namePart type="family">Bergen</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2024-05</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the 3rd Annual Meeting of the Special Interest Group on Under-resourced Languages @ LREC-COLING 2024</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Maite</namePart>
            <namePart type="family">Melero</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Sakriani</namePart>
            <namePart type="family">Sakti</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Claudia</namePart>
            <namePart type="family">Soria</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>ELRA and ICCL</publisher>
            <place>
                <placeTerm type="text">Torino, Italia</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>How should text dataset sizes be compared across languages? Even for content-matched (parallel) corpora, UTF-8 encoded text can require a dramatically different number of bytes for different languages. In our work, we define the byte premium between two languages as the ratio of bytes used to encode content-matched text in those languages. We compute byte premiums for 1155 languages, and we use linear regressions to estimate byte premiums for other languages. We release a tool to obtain byte premiums for any two languages, enabling comparisons of dataset sizes across languages for more equitable multilingual model development and data practices.</abstract>
    <identifier type="citekey">arnett-etal-2024-bit</identifier>
    <location>
        <url>https://aclanthology.org/2024.sigul-1.1</url>
    </location>
    <part>
        <date>2024-05</date>
        <extent unit="page">
            <start>1</start>
            <end>9</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T A Bit of a Problem: Measurement Disparities in Dataset Sizes across Languages
%A Arnett, Catherine
%A Chang, Tyler A.
%A Bergen, Benjamin
%Y Melero, Maite
%Y Sakti, Sakriani
%Y Soria, Claudia
%S Proceedings of the 3rd Annual Meeting of the Special Interest Group on Under-resourced Languages @ LREC-COLING 2024
%D 2024
%8 May
%I ELRA and ICCL
%C Torino, Italia
%F arnett-etal-2024-bit
%X How should text dataset sizes be compared across languages? Even for content-matched (parallel) corpora, UTF-8 encoded text can require a dramatically different number of bytes for different languages. In our work, we define the byte premium between two languages as the ratio of bytes used to encode content-matched text in those languages. We compute byte premiums for 1155 languages, and we use linear regressions to estimate byte premiums for other languages. We release a tool to obtain byte premiums for any two languages, enabling comparisons of dataset sizes across languages for more equitable multilingual model development and data practices.
%U https://aclanthology.org/2024.sigul-1.1
%P 1-9

Markdown (Informal)

[A Bit of a Problem: Measurement Disparities in Dataset Sizes across Languages](https://aclanthology.org/2024.sigul-1.1) (Arnett et al., SIGUL-WS 2024)

• A Bit of a Problem: Measurement Disparities in Dataset Sizes across Languages (Arnett et al., SIGUL-WS 2024)

ACL

• Catherine Arnett, Tyler A. Chang, and Benjamin Bergen. 2024. A Bit of a Problem: Measurement Disparities in Dataset Sizes across Languages. In Proceedings of the 3rd Annual Meeting of the Special Interest Group on Under-resourced Languages @ LREC-COLING 2024, pages 1–9, Torino, Italia. ELRA and ICCL.