<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="labeau-allauzen-2017-character">
    <titleInfo>
        <title>Character and Subword-Based Word Representation for Neural Language Modeling Prediction</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Matthieu</namePart>
        <namePart type="family">Labeau</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Alexandre</namePart>
        <namePart type="family">Allauzen</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2017-09</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the First Workshop on Subword and Character Level Models in NLP</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Manaal</namePart>
            <namePart type="family">Faruqui</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Hinrich</namePart>
            <namePart type="family">Schuetze</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Isabel</namePart>
            <namePart type="family">Trancoso</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Yadollah</namePart>
            <namePart type="family">Yaghoobzadeh</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Copenhagen, Denmark</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>Most of neural language models use different kinds of embeddings for word prediction. While word embeddings can be associated to each word in the vocabulary or derived from characters as well as factored morphological decomposition, these word representations are mainly used to parametrize the input, i.e. the context of prediction. This work investigates the effect of using subword units (character and factored morphological decomposition) to build output representations for neural language modeling. We present a case study on Czech, a morphologically-rich language, experimenting with different input and output representations. When working with the full training vocabulary, despite unstable training, our experiments show that augmenting the output word representations with character-based embeddings can significantly improve the performance of the model. Moreover, reducing the size of the output look-up table, to let the character-based embeddings represent rare words, brings further improvement.</abstract>
    <identifier type="citekey">labeau-allauzen-2017-character</identifier>
    <identifier type="doi">10.18653/v1/W17-4101</identifier>
    <location>
        <url>https://aclanthology.org/W17-4101</url>
    </location>
    <part>
        <date>2017-09</date>
        <extent unit="page">
            <start>1</start>
            <end>13</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Character and Subword-Based Word Representation for Neural Language Modeling Prediction
%A Labeau, Matthieu
%A Allauzen, Alexandre
%Y Faruqui, Manaal
%Y Schuetze, Hinrich
%Y Trancoso, Isabel
%Y Yaghoobzadeh, Yadollah
%S Proceedings of the First Workshop on Subword and Character Level Models in NLP
%D 2017
%8 September
%I Association for Computational Linguistics
%C Copenhagen, Denmark
%F labeau-allauzen-2017-character
%X Most of neural language models use different kinds of embeddings for word prediction. While word embeddings can be associated to each word in the vocabulary or derived from characters as well as factored morphological decomposition, these word representations are mainly used to parametrize the input, i.e. the context of prediction. This work investigates the effect of using subword units (character and factored morphological decomposition) to build output representations for neural language modeling. We present a case study on Czech, a morphologically-rich language, experimenting with different input and output representations. When working with the full training vocabulary, despite unstable training, our experiments show that augmenting the output word representations with character-based embeddings can significantly improve the performance of the model. Moreover, reducing the size of the output look-up table, to let the character-based embeddings represent rare words, brings further improvement.
%R 10.18653/v1/W17-4101
%U https://aclanthology.org/W17-4101
%U https://doi.org/10.18653/v1/W17-4101
%P 1-13

Markdown (Informal)

[Character and Subword-Based Word Representation for Neural Language Modeling Prediction](https://aclanthology.org/W17-4101) (Labeau & Allauzen, SCLeM 2017)

• Character and Subword-Based Word Representation for Neural Language Modeling Prediction (Labeau & Allauzen, SCLeM 2017)

ACL

• Matthieu Labeau and Alexandre Allauzen. 2017. Character and Subword-Based Word Representation for Neural Language Modeling Prediction. In Proceedings of the First Workshop on Subword and Character Level Models in NLP, pages 1–13, Copenhagen, Denmark. Association for Computational Linguistics.