<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="tanvir-etal-2021-estbert">
    <titleInfo>
        <title>EstBERT: A Pretrained Language-Specific BERT for Estonian</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Hasan</namePart>
        <namePart type="family">Tanvir</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Claudia</namePart>
        <namePart type="family">Kittask</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Sandra</namePart>
        <namePart type="family">Eiche</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Kairit</namePart>
        <namePart type="family">Sirts</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2021-may 31–2 jun</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the 23rd Nordic Conference on Computational Linguistics (NoDaLiDa)</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Simon</namePart>
            <namePart type="family">Dobnik</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Lilja</namePart>
            <namePart type="family">Øvrelid</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Linköping University Electronic Press, Sweden</publisher>
            <place>
                <placeTerm type="text">Reykjavik, Iceland (Online)</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>This paper presents EstBERT, a large pretrained transformer-based language-specific BERT model for Estonian. Recent work has evaluated multilingual BERT models on Estonian tasks and found them to outperform the baselines. Still, based on existing studies on other languages, a language-specific BERT model is expected to improve over the multilingual ones. We first describe the EstBERT pretraining process and then present the models’ results based on the finetuned EstBERT for multiple NLP tasks, including POS and morphological tagging, dependency parsing, named entity recognition and text classification. The evaluation results show that the models based on EstBERT outperform multilingual BERT models on five tasks out of seven, providing further evidence towards a view that training language-specific BERT models are still useful, even when multilingual models are available.</abstract>
    <identifier type="citekey">tanvir-etal-2021-estbert</identifier>
    <location>
        <url>https://aclanthology.org/2021.nodalida-main.2</url>
    </location>
    <part>
        <date>2021-may 31–2 jun</date>
        <extent unit="page">
            <start>11</start>
            <end>19</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T EstBERT: A Pretrained Language-Specific BERT for Estonian
%A Tanvir, Hasan
%A Kittask, Claudia
%A Eiche, Sandra
%A Sirts, Kairit
%Y Dobnik, Simon
%Y Øvrelid, Lilja
%S Proceedings of the 23rd Nordic Conference on Computational Linguistics (NoDaLiDa)
%D 2021
%8 may 31–2 jun
%I Linköping University Electronic Press, Sweden
%C Reykjavik, Iceland (Online)
%F tanvir-etal-2021-estbert
%X This paper presents EstBERT, a large pretrained transformer-based language-specific BERT model for Estonian. Recent work has evaluated multilingual BERT models on Estonian tasks and found them to outperform the baselines. Still, based on existing studies on other languages, a language-specific BERT model is expected to improve over the multilingual ones. We first describe the EstBERT pretraining process and then present the models’ results based on the finetuned EstBERT for multiple NLP tasks, including POS and morphological tagging, dependency parsing, named entity recognition and text classification. The evaluation results show that the models based on EstBERT outperform multilingual BERT models on five tasks out of seven, providing further evidence towards a view that training language-specific BERT models are still useful, even when multilingual models are available.
%U https://aclanthology.org/2021.nodalida-main.2
%P 11-19

Markdown (Informal)

[EstBERT: A Pretrained Language-Specific BERT for Estonian](https://aclanthology.org/2021.nodalida-main.2) (Tanvir et al., NoDaLiDa 2021)

• EstBERT: A Pretrained Language-Specific BERT for Estonian (Tanvir et al., NoDaLiDa 2021)

ACL

• Hasan Tanvir, Claudia Kittask, Sandra Eiche, and Kairit Sirts. 2021. EstBERT: A Pretrained Language-Specific BERT for Estonian. In Proceedings of the 23rd Nordic Conference on Computational Linguistics (NoDaLiDa), pages 11–19, Reykjavik, Iceland (Online). Linköping University Electronic Press, Sweden.