<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="salle-villavicencio-2018-restricted">
    <titleInfo>
        <title>Restricted Recurrent Neural Tensor Networks: Exploiting Word Frequency and Compositionality</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Alexandre</namePart>
        <namePart type="family">Salle</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Aline</namePart>
        <namePart type="family">Villavicencio</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2018-07</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Iryna</namePart>
            <namePart type="family">Gurevych</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Yusuke</namePart>
            <namePart type="family">Miyao</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Melbourne, Australia</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>Increasing the capacity of recurrent neural networks (RNN) usually involves augmenting the size of the hidden layer, with significant increase of computational cost. Recurrent neural tensor networks (RNTN) increase capacity using distinct hidden layer weights for each word, but with greater costs in memory usage. In this paper, we introduce restricted recurrent neural tensor networks (r-RNTN) which reserve distinct hidden layer weights for frequent vocabulary words while sharing a single set of weights for infrequent words. Perplexity evaluations show that for fixed hidden layer sizes, r-RNTNs improve language model performance over RNNs using only a small fraction of the parameters of unrestricted RNTNs. These results hold for r-RNTNs using Gated Recurrent Units and Long Short-Term Memory.</abstract>
    <identifier type="citekey">salle-villavicencio-2018-restricted</identifier>
    <identifier type="doi">10.18653/v1/P18-2002</identifier>
    <location>
        <url>https://aclanthology.org/P18-2002</url>
    </location>
    <part>
        <date>2018-07</date>
        <extent unit="page">
            <start>8</start>
            <end>13</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Restricted Recurrent Neural Tensor Networks: Exploiting Word Frequency and Compositionality
%A Salle, Alexandre
%A Villavicencio, Aline
%Y Gurevych, Iryna
%Y Miyao, Yusuke
%S Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)
%D 2018
%8 July
%I Association for Computational Linguistics
%C Melbourne, Australia
%F salle-villavicencio-2018-restricted
%X Increasing the capacity of recurrent neural networks (RNN) usually involves augmenting the size of the hidden layer, with significant increase of computational cost. Recurrent neural tensor networks (RNTN) increase capacity using distinct hidden layer weights for each word, but with greater costs in memory usage. In this paper, we introduce restricted recurrent neural tensor networks (r-RNTN) which reserve distinct hidden layer weights for frequent vocabulary words while sharing a single set of weights for infrequent words. Perplexity evaluations show that for fixed hidden layer sizes, r-RNTNs improve language model performance over RNNs using only a small fraction of the parameters of unrestricted RNTNs. These results hold for r-RNTNs using Gated Recurrent Units and Long Short-Term Memory.
%R 10.18653/v1/P18-2002
%U https://aclanthology.org/P18-2002
%U https://doi.org/10.18653/v1/P18-2002
%P 8-13

Markdown (Informal)

[Restricted Recurrent Neural Tensor Networks: Exploiting Word Frequency and Compositionality](https://aclanthology.org/P18-2002) (Salle & Villavicencio, ACL 2018)

• Restricted Recurrent Neural Tensor Networks: Exploiting Word Frequency and Compositionality (Salle & Villavicencio, ACL 2018)

ACL

• Alexandre Salle and Aline Villavicencio. 2018. Restricted Recurrent Neural Tensor Networks: Exploiting Word Frequency and Compositionality. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 8–13, Melbourne, Australia. Association for Computational Linguistics.