<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="watcharawittayakul-etal-2018-dual">
    <titleInfo>
        <title>Dual Fixed-Size Ordinally Forgetting Encoding (FOFE) for Competitive Neural Language Models</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Sedtawut</namePart>
        <namePart type="family">Watcharawittayakul</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Mingbin</namePart>
        <namePart type="family">Xu</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Hui</namePart>
        <namePart type="family">Jiang</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2018-oct-nov</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Ellen</namePart>
            <namePart type="family">Riloff</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">David</namePart>
            <namePart type="family">Chiang</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Julia</namePart>
            <namePart type="family">Hockenmaier</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Jun’ichi</namePart>
            <namePart type="family">Tsujii</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Brussels, Belgium</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>In this paper, we propose a new approach to employ the fixed-size ordinally-forgetting encoding (FOFE) (Zhang et al., 2015b) in neural languages modelling, called dual-FOFE. The main idea of dual-FOFE is that it allows to use two different forgetting factors so that it can avoid the trade-off in choosing either a small or large values for the single forgetting factor. In our experiments, we have compared the dual-FOFE based neural network language models (NNLM) against the original FOFE counterparts and various traditional NNLMs. Our results on the challenging Google Billion word corpus show that both FOFE and dual FOFE yield very strong performance while significantly reducing the computational complexity over other NNLMs. Furthermore, the proposed dual-FOFE method further gives over 10% improvement in perplexity over the original FOFE model.</abstract>
    <identifier type="citekey">watcharawittayakul-etal-2018-dual</identifier>
    <identifier type="doi">10.18653/v1/D18-1502</identifier>
    <location>
        <url>https://aclanthology.org/D18-1502</url>
    </location>
    <part>
        <date>2018-oct-nov</date>
        <extent unit="page">
            <start>4725</start>
            <end>4730</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Dual Fixed-Size Ordinally Forgetting Encoding (FOFE) for Competitive Neural Language Models
%A Watcharawittayakul, Sedtawut
%A Xu, Mingbin
%A Jiang, Hui
%Y Riloff, Ellen
%Y Chiang, David
%Y Hockenmaier, Julia
%Y Tsujii, Jun’ichi
%S Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing
%D 2018
%8 oct nov
%I Association for Computational Linguistics
%C Brussels, Belgium
%F watcharawittayakul-etal-2018-dual
%X In this paper, we propose a new approach to employ the fixed-size ordinally-forgetting encoding (FOFE) (Zhang et al., 2015b) in neural languages modelling, called dual-FOFE. The main idea of dual-FOFE is that it allows to use two different forgetting factors so that it can avoid the trade-off in choosing either a small or large values for the single forgetting factor. In our experiments, we have compared the dual-FOFE based neural network language models (NNLM) against the original FOFE counterparts and various traditional NNLMs. Our results on the challenging Google Billion word corpus show that both FOFE and dual FOFE yield very strong performance while significantly reducing the computational complexity over other NNLMs. Furthermore, the proposed dual-FOFE method further gives over 10% improvement in perplexity over the original FOFE model.
%R 10.18653/v1/D18-1502
%U https://aclanthology.org/D18-1502
%U https://doi.org/10.18653/v1/D18-1502
%P 4725-4730

Markdown (Informal)

[Dual Fixed-Size Ordinally Forgetting Encoding (FOFE) for Competitive Neural Language Models](https://aclanthology.org/D18-1502) (Watcharawittayakul et al., EMNLP 2018)

• Dual Fixed-Size Ordinally Forgetting Encoding (FOFE) for Competitive Neural Language Models (Watcharawittayakul et al., EMNLP 2018)

ACL

• Sedtawut Watcharawittayakul, Mingbin Xu, and Hui Jiang. 2018. Dual Fixed-Size Ordinally Forgetting Encoding (FOFE) for Competitive Neural Language Models. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 4725–4730, Brussels, Belgium. Association for Computational Linguistics.