<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="znotins-2026-pretraining">
    <titleInfo>
        <title>Pretraining and Benchmarking Modern Encoders for Latvian</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Arturs</namePart>
        <namePart type="family">Znotins</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2026-03</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the Second Workshop on Language Models for Low-Resource Languages (LoResLM 2026)</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Hansi</namePart>
            <namePart type="family">Hettiarachchi</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Tharindu</namePart>
            <namePart type="family">Ranasinghe</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Alistair</namePart>
            <namePart type="family">Plum</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Paul</namePart>
            <namePart type="family">Rayson</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Ruslan</namePart>
            <namePart type="family">Mitkov</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Mohamed</namePart>
            <namePart type="family">Gaber</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Damith</namePart>
            <namePart type="family">Premasiri</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Fiona</namePart>
            <namePart type="given">Anting</namePart>
            <namePart type="family">Tan</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Lasitha</namePart>
            <namePart type="family">Uyangodage</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Rabat, Morocco</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
        <identifier type="isbn">979-8-89176-377-7</identifier>
    </relatedItem>
    <abstract>Encoder-only transformers remain essential for practical NLP tasks. While recent advances in multilingual models have improved cross-lingual capabilities, low-resource languages such as Latvian remain underrepresented in pretraining corpora, and few monolingual Latvian encoders currently exist. We address this gap by pretraining a suite of Latvian-specific encoders based on RoBERTa, DeBERTaV3, and ModernBERT architectures, including long-context variants, and evaluating them on a comprehensive Latvian benchmark suite. Our models are competitive with existing monolingual and multilingual encoders while benefiting from recent architectural and efficiency advances. Our best model, lv-deberta-base (111M parameters), achieves the strongest overall performance, outperforming larger multilingual baselines and prior Latvian-specific encoders. We release all pretrained models and evaluation resources to support further research and practical applications in Latvian NLP.</abstract>
    <identifier type="citekey">znotins-2026-pretraining</identifier>
    <location>
        <url>https://aclanthology.org/2026.loreslm-1.40/</url>
    </location>
    <part>
        <date>2026-03</date>
        <extent unit="page">
            <start>461</start>
            <end>470</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Pretraining and Benchmarking Modern Encoders for Latvian
%A Znotins, Arturs
%Y Hettiarachchi, Hansi
%Y Ranasinghe, Tharindu
%Y Plum, Alistair
%Y Rayson, Paul
%Y Mitkov, Ruslan
%Y Gaber, Mohamed
%Y Premasiri, Damith
%Y Tan, Fiona Anting
%Y Uyangodage, Lasitha
%S Proceedings of the Second Workshop on Language Models for Low-Resource Languages (LoResLM 2026)
%D 2026
%8 March
%I Association for Computational Linguistics
%C Rabat, Morocco
%@ 979-8-89176-377-7
%F znotins-2026-pretraining
%X Encoder-only transformers remain essential for practical NLP tasks. While recent advances in multilingual models have improved cross-lingual capabilities, low-resource languages such as Latvian remain underrepresented in pretraining corpora, and few monolingual Latvian encoders currently exist. We address this gap by pretraining a suite of Latvian-specific encoders based on RoBERTa, DeBERTaV3, and ModernBERT architectures, including long-context variants, and evaluating them on a comprehensive Latvian benchmark suite. Our models are competitive with existing monolingual and multilingual encoders while benefiting from recent architectural and efficiency advances. Our best model, lv-deberta-base (111M parameters), achieves the strongest overall performance, outperforming larger multilingual baselines and prior Latvian-specific encoders. We release all pretrained models and evaluation resources to support further research and practical applications in Latvian NLP.
%U https://aclanthology.org/2026.loreslm-1.40/
%P 461-470

Markdown (Informal)

[Pretraining and Benchmarking Modern Encoders for Latvian](https://aclanthology.org/2026.loreslm-1.40/) (Znotins, LoResLM 2026)

• Pretraining and Benchmarking Modern Encoders for Latvian (Znotins, LoResLM 2026)

ACL

• Arturs Znotins. 2026. Pretraining and Benchmarking Modern Encoders for Latvian. In Proceedings of the Second Workshop on Language Models for Low-Resource Languages (LoResLM 2026), pages 461–470, Rabat, Morocco. Association for Computational Linguistics.