<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="montariol-allauzen-2019-empirical">
    <titleInfo>
        <title>Empirical Study of Diachronic Word Embeddings for Scarce Data</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Syrielle</namePart>
        <namePart type="family">Montariol</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Alexandre</namePart>
        <namePart type="family">Allauzen</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2019-09</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the International Conference on Recent Advances in Natural Language Processing (RANLP 2019)</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Ruslan</namePart>
            <namePart type="family">Mitkov</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Galia</namePart>
            <namePart type="family">Angelova</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>INCOMA Ltd.</publisher>
            <place>
                <placeTerm type="text">Varna, Bulgaria</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>Word meaning change can be inferred from drifts of time-varying word embeddings. However, temporal data may be too sparse to build robust word embeddings and to discriminate significant drifts from noise. In this paper, we compare three models to learn diachronic word embeddings on scarce data: incremental updating of a Skip-Gram from Kim et al. (2014), dynamic filtering from Bamler &amp; Mandt (2017), and dynamic Bernoulli embeddings from Rudolph &amp; Blei (2018). In particular, we study the performance of different initialisation schemes and emphasise what characteristics of each model are more suitable to data scarcity, relying on the distribution of detected drifts. Finally, we regularise the loss of these models to better adapt to scarce data.</abstract>
    <identifier type="citekey">montariol-allauzen-2019-empirical</identifier>
    <identifier type="doi">10.26615/978-954-452-056-4_092</identifier>
    <location>
        <url>https://aclanthology.org/R19-1092</url>
    </location>
    <part>
        <date>2019-09</date>
        <extent unit="page">
            <start>795</start>
            <end>803</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Empirical Study of Diachronic Word Embeddings for Scarce Data
%A Montariol, Syrielle
%A Allauzen, Alexandre
%Y Mitkov, Ruslan
%Y Angelova, Galia
%S Proceedings of the International Conference on Recent Advances in Natural Language Processing (RANLP 2019)
%D 2019
%8 September
%I INCOMA Ltd.
%C Varna, Bulgaria
%F montariol-allauzen-2019-empirical
%X Word meaning change can be inferred from drifts of time-varying word embeddings. However, temporal data may be too sparse to build robust word embeddings and to discriminate significant drifts from noise. In this paper, we compare three models to learn diachronic word embeddings on scarce data: incremental updating of a Skip-Gram from Kim et al. (2014), dynamic filtering from Bamler & Mandt (2017), and dynamic Bernoulli embeddings from Rudolph & Blei (2018). In particular, we study the performance of different initialisation schemes and emphasise what characteristics of each model are more suitable to data scarcity, relying on the distribution of detected drifts. Finally, we regularise the loss of these models to better adapt to scarce data.
%R 10.26615/978-954-452-056-4_092
%U https://aclanthology.org/R19-1092
%U https://doi.org/10.26615/978-954-452-056-4_092
%P 795-803

Markdown (Informal)

[Empirical Study of Diachronic Word Embeddings for Scarce Data](https://aclanthology.org/R19-1092) (Montariol & Allauzen, RANLP 2019)

• Empirical Study of Diachronic Word Embeddings for Scarce Data (Montariol & Allauzen, RANLP 2019)

ACL

• Syrielle Montariol and Alexandre Allauzen. 2019. Empirical Study of Diachronic Word Embeddings for Scarce Data. In Proceedings of the International Conference on Recent Advances in Natural Language Processing (RANLP 2019), pages 795–803, Varna, Bulgaria. INCOMA Ltd..