<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="de-villiers-eiselen-2026-benchmarking">
    <titleInfo>
        <title>Benchmarking Text Embedding Models for South African Languages</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Ockert</namePart>
        <namePart type="family">de Villiers</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Roald</namePart>
        <namePart type="family">Eiselen</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2026-05</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of Resources for African Indigenous Languages (RAIL) 2026 @ LREC 2026</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Muzi</namePart>
            <namePart type="family">Matfunjwa</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Mmasibidi</namePart>
            <namePart type="family">Setaka</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Rooweither</namePart>
            <namePart type="family">Mabuya</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Menno</namePart>
            <namePart type="family">van Zaanen</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>ELRA Language Resources Association (ELRA)</publisher>
            <place>
                <placeTerm type="text">Palma, Mallorca (Spain)</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>In this work we introduce a collection of monolingual embedding models for ten South African languages in four different architectures. To determine the quality of the embedding models we evaluate the embeddings on two sequence-labelling tasks, namely Part-of-Speech (POS) tagging and Named Entity Recognition (NER). Languages are grouped into conjunctive (isiNdebele, isiXhosa, isiZulu, and Siswati), disjunctive (Sepedi, Sesotho, Setswana, Tshivenḓa, and Xitsonga), and Afrikaans to establish the influence of training data set size and typology on the quality of the different embeddings. To isolate representation effects we train BiLSTM-CRF taggers, while keeping the architecture, data splits, and training budget fixed, varying only the input imbedding representations, namely GloVe, fastText, Flair, and RoBERTa. In our experiments, GloVe lags behind fastText, Flair, and the transformer-based models, confirming that static word-level vectors are less suited to morphologically complex, low-resource languages. Subword-aware embeddings such as fastText remain a reliable and computationally efficient baseline, while Flair is the most competitive overall across both POS tagging and NER tasks.</abstract>
    <identifier type="citekey">de-villiers-eiselen-2026-benchmarking</identifier>
    <identifier type="doi">10.63317/28nicfxsyx92</identifier>
    <location>
        <url>https://aclanthology.org/2026.rail-1.5/</url>
    </location>
    <part>
        <date>2026-05</date>
        <extent unit="page">
            <start>41</start>
            <end>51</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Benchmarking Text Embedding Models for South African Languages
%A de Villiers, Ockert
%A Eiselen, Roald
%Y Matfunjwa, Muzi
%Y Setaka, Mmasibidi
%Y Mabuya, Rooweither
%Y van Zaanen, Menno
%S Proceedings of Resources for African Indigenous Languages (RAIL) 2026 @ LREC 2026
%D 2026
%8 May
%I ELRA Language Resources Association (ELRA)
%C Palma, Mallorca (Spain)
%F de-villiers-eiselen-2026-benchmarking
%X In this work we introduce a collection of monolingual embedding models for ten South African languages in four different architectures. To determine the quality of the embedding models we evaluate the embeddings on two sequence-labelling tasks, namely Part-of-Speech (POS) tagging and Named Entity Recognition (NER). Languages are grouped into conjunctive (isiNdebele, isiXhosa, isiZulu, and Siswati), disjunctive (Sepedi, Sesotho, Setswana, Tshivenḓa, and Xitsonga), and Afrikaans to establish the influence of training data set size and typology on the quality of the different embeddings. To isolate representation effects we train BiLSTM-CRF taggers, while keeping the architecture, data splits, and training budget fixed, varying only the input imbedding representations, namely GloVe, fastText, Flair, and RoBERTa. In our experiments, GloVe lags behind fastText, Flair, and the transformer-based models, confirming that static word-level vectors are less suited to morphologically complex, low-resource languages. Subword-aware embeddings such as fastText remain a reliable and computationally efficient baseline, while Flair is the most competitive overall across both POS tagging and NER tasks.
%R 10.63317/28nicfxsyx92
%U https://aclanthology.org/2026.rail-1.5/
%U https://doi.org/10.63317/28nicfxsyx92
%P 41-51

Markdown (Informal)

[Benchmarking Text Embedding Models for South African Languages](https://aclanthology.org/2026.rail-1.5/) (de Villiers & Eiselen, RAIL 2026)

• Benchmarking Text Embedding Models for South African Languages (de Villiers & Eiselen, RAIL 2026)

ACL

• Ockert de Villiers and Roald Eiselen. 2026. Benchmarking Text Embedding Models for South African Languages. In Proceedings of Resources for African Indigenous Languages (RAIL) 2026 @ LREC 2026, pages 41–51, Palma, Mallorca (Spain). ELRA Language Resources Association (ELRA).