<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="scherrer-ljubesic-2020-helju">
    <titleInfo>
        <title>HeLju@VarDial 2020: Social Media Variety Geolocation with BERT Models</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Yves</namePart>
        <namePart type="family">Scherrer</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Nikola</namePart>
        <namePart type="family">Ljubešić</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2020-12</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the 7th Workshop on NLP for Similar Languages, Varieties and Dialects</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Marcos</namePart>
            <namePart type="family">Zampieri</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Preslav</namePart>
            <namePart type="family">Nakov</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Nikola</namePart>
            <namePart type="family">Ljubešić</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Jörg</namePart>
            <namePart type="family">Tiedemann</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Yves</namePart>
            <namePart type="family">Scherrer</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>International Committee on Computational Linguistics (ICCL)</publisher>
            <place>
                <placeTerm type="text">Barcelona, Spain (Online)</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>This paper describes the Helsinki-Ljubljana contribution to the VarDial shared task on social media variety geolocation. Our solutions are based on the BERT Transformer models, the constrained versions of our models reaching 1st place in two subtasks and 3rd place in one subtask, while our unconstrained models outperform all the constrained systems by a large margin. We show in our analyses that Transformer-based models outperform traditional models by far, and that improvements obtained by pre-training models on large quantities of (mostly standard) text are significant, but not drastic, with single-language models also outperforming multilingual models. Our manual analysis shows that two types of signals are the most crucial for a (mis)prediction: named entities and dialectal features, both of which are handled well by our models.</abstract>
    <identifier type="citekey">scherrer-ljubesic-2020-helju</identifier>
    <location>
        <url>https://aclanthology.org/2020.vardial-1.19</url>
    </location>
    <part>
        <date>2020-12</date>
        <extent unit="page">
            <start>202</start>
            <end>211</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T HeLju@VarDial 2020: Social Media Variety Geolocation with BERT Models
%A Scherrer, Yves
%A Ljubešić, Nikola
%Y Zampieri, Marcos
%Y Nakov, Preslav
%Y Ljubešić, Nikola
%Y Tiedemann, Jörg
%Y Scherrer, Yves
%S Proceedings of the 7th Workshop on NLP for Similar Languages, Varieties and Dialects
%D 2020
%8 December
%I International Committee on Computational Linguistics (ICCL)
%C Barcelona, Spain (Online)
%F scherrer-ljubesic-2020-helju
%X This paper describes the Helsinki-Ljubljana contribution to the VarDial shared task on social media variety geolocation. Our solutions are based on the BERT Transformer models, the constrained versions of our models reaching 1st place in two subtasks and 3rd place in one subtask, while our unconstrained models outperform all the constrained systems by a large margin. We show in our analyses that Transformer-based models outperform traditional models by far, and that improvements obtained by pre-training models on large quantities of (mostly standard) text are significant, but not drastic, with single-language models also outperforming multilingual models. Our manual analysis shows that two types of signals are the most crucial for a (mis)prediction: named entities and dialectal features, both of which are handled well by our models.
%U https://aclanthology.org/2020.vardial-1.19
%P 202-211

Markdown (Informal)

[HeLju@VarDial 2020: Social Media Variety Geolocation with BERT Models](https://aclanthology.org/2020.vardial-1.19) (Scherrer & Ljubešić, VarDial 2020)

• HeLju@VarDial 2020: Social Media Variety Geolocation with BERT Models (Scherrer & Ljubešić, VarDial 2020)

ACL

• Yves Scherrer and Nikola Ljubešić. 2020. HeLju@VarDial 2020: Social Media Variety Geolocation with BERT Models. In Proceedings of the 7th Workshop on NLP for Similar Languages, Varieties and Dialects, pages 202–211, Barcelona, Spain (Online). International Committee on Computational Linguistics (ICCL).