<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="ploeger-etal-2026-principled">
    <titleInfo>
        <title>A Principled Framework for Evaluating on Typologically Diverse Languages</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Esther</namePart>
        <namePart type="family">Ploeger</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Wessel</namePart>
        <namePart type="family">Poelman</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Andreas</namePart>
        <namePart type="given">Holck</namePart>
        <namePart type="family">Høeg-Petersen</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Anders</namePart>
        <namePart type="family">Schlichtkrull</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Miryam</namePart>
        <namePart type="family">de Lhoneux</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Johannes</namePart>
        <namePart type="family">Bjerva</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2026-03</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <genre authority="bibutilsgt">journal article</genre>
    <relatedItem type="host">
        <titleInfo>
            <title>Computational Linguistics</title>
        </titleInfo>
        <originInfo>
            <issuance>continuing</issuance>
            <publisher>MIT Press</publisher>
            <place>
                <placeTerm type="text">Cambridge, MA</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">periodical</genre>
        <genre authority="bibutilsgt">academic journal</genre>
    </relatedItem>
    <abstract>Beyond individual languages, multilingual natural language processing (NLP) research increasingly aims to develop models that perform well across languages generally. However, evaluating these systems on all the world’s languages is practically infeasible. To attain generalizability, representative language sampling is essential. Previous work argues that generalizable multilingual evaluation sets should contain languages with diverse typological properties. However, “typologically diverse” language samples have been found to vary considerably in this regard, and popular sampling methods are flawed and inconsistent. We present a language sampling framework for selecting highly typologically diverse languages given a sampling frame, informed by language typology. We compare sampling methods with a range of metrics and find that our systematic methods consistently retrieve more typologically diverse language selections than previous methods in NLP. Moreover, we provide evidence that this affects generalizability in multilingual model evaluation, emphasizing the importance of diverse language sampling in NLP evaluation.</abstract>
    <identifier type="citekey">ploeger-etal-2026-principled</identifier>
    <identifier type="doi">10.1162/coli.a.577</identifier>
    <location>
        <url>https://aclanthology.org/2026.cl-1.7/</url>
    </location>
    <part>
        <date>2026-03</date>
        <detail type="volume"><number>52</number></detail>
        <detail type="issue"><number>1</number></detail>
        <extent unit="page">
            <start>237</start>
            <end>269</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Journal Article
%T A Principled Framework for Evaluating on Typologically Diverse Languages
%A Ploeger, Esther
%A Poelman, Wessel
%A Høeg-Petersen, Andreas Holck
%A Schlichtkrull, Anders
%A de Lhoneux, Miryam
%A Bjerva, Johannes
%J Computational Linguistics
%D 2026
%8 March
%V 52
%N 1
%I MIT Press
%C Cambridge, MA
%F ploeger-etal-2026-principled
%X Beyond individual languages, multilingual natural language processing (NLP) research increasingly aims to develop models that perform well across languages generally. However, evaluating these systems on all the world’s languages is practically infeasible. To attain generalizability, representative language sampling is essential. Previous work argues that generalizable multilingual evaluation sets should contain languages with diverse typological properties. However, “typologically diverse” language samples have been found to vary considerably in this regard, and popular sampling methods are flawed and inconsistent. We present a language sampling framework for selecting highly typologically diverse languages given a sampling frame, informed by language typology. We compare sampling methods with a range of metrics and find that our systematic methods consistently retrieve more typologically diverse language selections than previous methods in NLP. Moreover, we provide evidence that this affects generalizability in multilingual model evaluation, emphasizing the importance of diverse language sampling in NLP evaluation.
%R 10.1162/coli.a.577
%U https://aclanthology.org/2026.cl-1.7/
%U https://doi.org/10.1162/coli.a.577
%P 237-269

Markdown (Informal)

[A Principled Framework for Evaluating on Typologically Diverse Languages](https://aclanthology.org/2026.cl-1.7/) (Ploeger et al., CL 2026)

• A Principled Framework for Evaluating on Typologically Diverse Languages (Ploeger et al., CL 2026)

ACL

• Esther Ploeger, Wessel Poelman, Andreas Holck Høeg-Petersen, Anders Schlichtkrull, Miryam de Lhoneux, and Johannes Bjerva. 2026. A Principled Framework for Evaluating on Typologically Diverse Languages. Computational Linguistics, 52(1):237–269.