<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="kobayashi-etal-2017-neural">
    <titleInfo>
        <title>A Neural Language Model for Dynamically Representing the Meanings of Unknown Words and Entities in a Discourse</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Sosuke</namePart>
        <namePart type="family">Kobayashi</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Naoaki</namePart>
        <namePart type="family">Okazaki</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Kentaro</namePart>
        <namePart type="family">Inui</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2017-11</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the Eighth International Joint Conference on Natural Language Processing (Volume 1: Long Papers)</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Greg</namePart>
            <namePart type="family">Kondrak</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Taro</namePart>
            <namePart type="family">Watanabe</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Asian Federation of Natural Language Processing</publisher>
            <place>
                <placeTerm type="text">Taipei, Taiwan</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>This study addresses the problem of identifying the meaning of unknown words or entities in a discourse with respect to the word embedding approaches used in neural language models. We proposed a method for on-the-fly construction and exploitation of word embeddings in both the input and output layers of a neural model by tracking contexts. This extends the dynamic entity representation used in Kobayashi et al. (2016) and incorporates a copy mechanism proposed independently by Gu et al. (2016) and Gulcehre et al. (2016). In addition, we construct a new task and dataset called Anonymized Language Modeling for evaluating the ability to capture word meanings while reading. Experiments conducted using our novel dataset show that the proposed variant of RNN language model outperformed the baseline model. Furthermore, the experiments also demonstrate that dynamic updates of an output layer help a model predict reappearing entities, whereas those of an input layer are effective to predict words following reappearing entities.</abstract>
    <identifier type="citekey">kobayashi-etal-2017-neural</identifier>
    <location>
        <url>https://aclanthology.org/I17-1048</url>
    </location>
    <part>
        <date>2017-11</date>
        <extent unit="page">
            <start>473</start>
            <end>483</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T A Neural Language Model for Dynamically Representing the Meanings of Unknown Words and Entities in a Discourse
%A Kobayashi, Sosuke
%A Okazaki, Naoaki
%A Inui, Kentaro
%Y Kondrak, Greg
%Y Watanabe, Taro
%S Proceedings of the Eighth International Joint Conference on Natural Language Processing (Volume 1: Long Papers)
%D 2017
%8 November
%I Asian Federation of Natural Language Processing
%C Taipei, Taiwan
%F kobayashi-etal-2017-neural
%X This study addresses the problem of identifying the meaning of unknown words or entities in a discourse with respect to the word embedding approaches used in neural language models. We proposed a method for on-the-fly construction and exploitation of word embeddings in both the input and output layers of a neural model by tracking contexts. This extends the dynamic entity representation used in Kobayashi et al. (2016) and incorporates a copy mechanism proposed independently by Gu et al. (2016) and Gulcehre et al. (2016). In addition, we construct a new task and dataset called Anonymized Language Modeling for evaluating the ability to capture word meanings while reading. Experiments conducted using our novel dataset show that the proposed variant of RNN language model outperformed the baseline model. Furthermore, the experiments also demonstrate that dynamic updates of an output layer help a model predict reappearing entities, whereas those of an input layer are effective to predict words following reappearing entities.
%U https://aclanthology.org/I17-1048
%P 473-483

Markdown (Informal)

[A Neural Language Model for Dynamically Representing the Meanings of Unknown Words and Entities in a Discourse](https://aclanthology.org/I17-1048) (Kobayashi et al., IJCNLP 2017)

• A Neural Language Model for Dynamically Representing the Meanings of Unknown Words and Entities in a Discourse (Kobayashi et al., IJCNLP 2017)

ACL

• Sosuke Kobayashi, Naoaki Okazaki, and Kentaro Inui. 2017. A Neural Language Model for Dynamically Representing the Meanings of Unknown Words and Entities in a Discourse. In Proceedings of the Eighth International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 473–483, Taipei, Taiwan. Asian Federation of Natural Language Processing.