<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="li-etal-2022-residual">
    <titleInfo>
        <title>Residual Learning of Neural Text Generation with n-gram Language Model</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Huayang</namePart>
        <namePart type="family">Li</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Deng</namePart>
        <namePart type="family">Cai</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Jin</namePart>
        <namePart type="family">Xu</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Taro</namePart>
        <namePart type="family">Watanabe</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2022-12</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Findings of the Association for Computational Linguistics: EMNLP 2022</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Yoav</namePart>
            <namePart type="family">Goldberg</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Zornitsa</namePart>
            <namePart type="family">Kozareva</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Yue</namePart>
            <namePart type="family">Zhang</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Abu Dhabi, United Arab Emirates</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>N-gram language models (LM) has been largely superseded by neural LMs as the latter exhibits better performance. However, we find that n-gram models can achieve satisfactory performance on a large proportion of testing cases, indicating they have already captured abundant knowledge of the language with relatively low computational cost. With this observation, we propose to learn a neural LM that fits the residual between an n-gram LM and the real-data distribution. The combination of n-gram LMs and neural LMs not only allows the neural part to focus on deeper understanding of the language, but also provides a flexible way to customize a LM by switching the underlying n-gram model without changing the neural model. Experimental results on three typical language tasks (i.e., language modeling, machine translation, and summarization) demonstrate that our approach attains additional performance gains over popular standalone neural models consistently. We also show that our approach allows for effective domain adaptation by simply switching to a domain-specific n-gram model, without any extra training.</abstract>
    <identifier type="citekey">li-etal-2022-residual</identifier>
    <identifier type="doi">10.18653/v1/2022.findings-emnlp.109</identifier>
    <location>
        <url>https://aclanthology.org/2022.findings-emnlp.109</url>
    </location>
    <part>
        <date>2022-12</date>
        <extent unit="page">
            <start>1523</start>
            <end>1533</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Residual Learning of Neural Text Generation with n-gram Language Model
%A Li, Huayang
%A Cai, Deng
%A Xu, Jin
%A Watanabe, Taro
%Y Goldberg, Yoav
%Y Kozareva, Zornitsa
%Y Zhang, Yue
%S Findings of the Association for Computational Linguistics: EMNLP 2022
%D 2022
%8 December
%I Association for Computational Linguistics
%C Abu Dhabi, United Arab Emirates
%F li-etal-2022-residual
%X N-gram language models (LM) has been largely superseded by neural LMs as the latter exhibits better performance. However, we find that n-gram models can achieve satisfactory performance on a large proportion of testing cases, indicating they have already captured abundant knowledge of the language with relatively low computational cost. With this observation, we propose to learn a neural LM that fits the residual between an n-gram LM and the real-data distribution. The combination of n-gram LMs and neural LMs not only allows the neural part to focus on deeper understanding of the language, but also provides a flexible way to customize a LM by switching the underlying n-gram model without changing the neural model. Experimental results on three typical language tasks (i.e., language modeling, machine translation, and summarization) demonstrate that our approach attains additional performance gains over popular standalone neural models consistently. We also show that our approach allows for effective domain adaptation by simply switching to a domain-specific n-gram model, without any extra training.
%R 10.18653/v1/2022.findings-emnlp.109
%U https://aclanthology.org/2022.findings-emnlp.109
%U https://doi.org/10.18653/v1/2022.findings-emnlp.109
%P 1523-1533

Markdown (Informal)

[Residual Learning of Neural Text Generation with n-gram Language Model](https://aclanthology.org/2022.findings-emnlp.109) (Li et al., Findings 2022)

• Residual Learning of Neural Text Generation with n-gram Language Model (Li et al., Findings 2022)

ACL

• Huayang Li, Deng Cai, Jin Xu, and Taro Watanabe. 2022. Residual Learning of Neural Text Generation with n-gram Language Model. In Findings of the Association for Computational Linguistics: EMNLP 2022, pages 1523–1533, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics.