<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="kunz-hardmeier-2019-entity">
    <titleInfo>
        <title>Entity Decisions in Neural Language Modelling: Approaches and Problems</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Jenny</namePart>
        <namePart type="family">Kunz</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Christian</namePart>
        <namePart type="family">Hardmeier</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2019-06</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the Second Workshop on Computational Models of Reference, Anaphora and Coreference</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Maciej</namePart>
            <namePart type="family">Ogrodniczuk</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Sameer</namePart>
            <namePart type="family">Pradhan</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Yulia</namePart>
            <namePart type="family">Grishina</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Vincent</namePart>
            <namePart type="family">Ng</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Minneapolis, USA</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>We explore different approaches to explicit entity modelling in language models (LM). We independently replicate two existing models in a controlled setup, introduce a simplified variant of one of the models and analyze their performance in direct comparison. Our results suggest that today’s models are limited as several stochastic variables make learning difficult. We show that the most challenging point in the systems is the decision if the next token is an entity token. The low precision and recall for this variable will lead to severe cascading errors. Our own simplified approach dispenses with the need for latent variables and improves the performance in the entity yes/no decision. A standard well-tuned baseline RNN-LM with a larger number of hidden units outperforms all entity-enabled LMs in terms of perplexity.</abstract>
    <identifier type="citekey">kunz-hardmeier-2019-entity</identifier>
    <identifier type="doi">10.18653/v1/W19-2803</identifier>
    <location>
        <url>https://aclanthology.org/W19-2803</url>
    </location>
    <part>
        <date>2019-06</date>
        <extent unit="page">
            <start>15</start>
            <end>19</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Entity Decisions in Neural Language Modelling: Approaches and Problems
%A Kunz, Jenny
%A Hardmeier, Christian
%Y Ogrodniczuk, Maciej
%Y Pradhan, Sameer
%Y Grishina, Yulia
%Y Ng, Vincent
%S Proceedings of the Second Workshop on Computational Models of Reference, Anaphora and Coreference
%D 2019
%8 June
%I Association for Computational Linguistics
%C Minneapolis, USA
%F kunz-hardmeier-2019-entity
%X We explore different approaches to explicit entity modelling in language models (LM). We independently replicate two existing models in a controlled setup, introduce a simplified variant of one of the models and analyze their performance in direct comparison. Our results suggest that today’s models are limited as several stochastic variables make learning difficult. We show that the most challenging point in the systems is the decision if the next token is an entity token. The low precision and recall for this variable will lead to severe cascading errors. Our own simplified approach dispenses with the need for latent variables and improves the performance in the entity yes/no decision. A standard well-tuned baseline RNN-LM with a larger number of hidden units outperforms all entity-enabled LMs in terms of perplexity.
%R 10.18653/v1/W19-2803
%U https://aclanthology.org/W19-2803
%U https://doi.org/10.18653/v1/W19-2803
%P 15-19

Markdown (Informal)

[Entity Decisions in Neural Language Modelling: Approaches and Problems](https://aclanthology.org/W19-2803) (Kunz & Hardmeier, CRAC 2019)

• Entity Decisions in Neural Language Modelling: Approaches and Problems (Kunz & Hardmeier, CRAC 2019)

ACL

• Jenny Kunz and Christian Hardmeier. 2019. Entity Decisions in Neural Language Modelling: Approaches and Problems. In Proceedings of the Second Workshop on Computational Models of Reference, Anaphora and Coreference, pages 15–19, Minneapolis, USA. Association for Computational Linguistics.