<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="wu-etal-2022-modeling">
    <titleInfo>
        <title>Modeling Context With Linear Attention for Scalable Document-Level Translation</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Zhaofeng</namePart>
        <namePart type="family">Wu</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Hao</namePart>
        <namePart type="family">Peng</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Nikolaos</namePart>
        <namePart type="family">Pappas</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Noah</namePart>
        <namePart type="given">A</namePart>
        <namePart type="family">Smith</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2022-12</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Findings of the Association for Computational Linguistics: EMNLP 2022</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Yoav</namePart>
            <namePart type="family">Goldberg</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Zornitsa</namePart>
            <namePart type="family">Kozareva</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Yue</namePart>
            <namePart type="family">Zhang</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Abu Dhabi, United Arab Emirates</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>Document-level machine translation leverages inter-sentence dependencies to produce more coherent and consistent translations. However, these models, predominantly based on transformers, are difficult to scale to long documents as their attention layers have quadratic complexity in the sequence length. Recent efforts on efficient attention improve scalability, but their effect on document translation remains unexplored. In this work, we investigate the efficacy of a recent linear attention model by Peng et al. (2021) on document translation and augment it with a sentential gate to promote a recency inductive bias. We evaluate the model on IWSLT 2015 and OpenSubtitles 2018 against the transformer, demonstrating substantially increased decoding speed on long sequences with similar or better BLEU scores. We show that sentential gating further improves translation quality on IWSLT.</abstract>
    <identifier type="citekey">wu-etal-2022-modeling</identifier>
    <identifier type="doi">10.18653/v1/2022.findings-emnlp.515</identifier>
    <location>
        <url>https://aclanthology.org/2022.findings-emnlp.515</url>
    </location>
    <part>
        <date>2022-12</date>
        <extent unit="page">
            <start>6931</start>
            <end>6939</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Modeling Context With Linear Attention for Scalable Document-Level Translation
%A Wu, Zhaofeng
%A Peng, Hao
%A Pappas, Nikolaos
%A Smith, Noah A.
%Y Goldberg, Yoav
%Y Kozareva, Zornitsa
%Y Zhang, Yue
%S Findings of the Association for Computational Linguistics: EMNLP 2022
%D 2022
%8 December
%I Association for Computational Linguistics
%C Abu Dhabi, United Arab Emirates
%F wu-etal-2022-modeling
%X Document-level machine translation leverages inter-sentence dependencies to produce more coherent and consistent translations. However, these models, predominantly based on transformers, are difficult to scale to long documents as their attention layers have quadratic complexity in the sequence length. Recent efforts on efficient attention improve scalability, but their effect on document translation remains unexplored. In this work, we investigate the efficacy of a recent linear attention model by Peng et al. (2021) on document translation and augment it with a sentential gate to promote a recency inductive bias. We evaluate the model on IWSLT 2015 and OpenSubtitles 2018 against the transformer, demonstrating substantially increased decoding speed on long sequences with similar or better BLEU scores. We show that sentential gating further improves translation quality on IWSLT.
%R 10.18653/v1/2022.findings-emnlp.515
%U https://aclanthology.org/2022.findings-emnlp.515
%U https://doi.org/10.18653/v1/2022.findings-emnlp.515
%P 6931-6939

Markdown (Informal)

[Modeling Context With Linear Attention for Scalable Document-Level Translation](https://aclanthology.org/2022.findings-emnlp.515) (Wu et al., Findings 2022)

• Modeling Context With Linear Attention for Scalable Document-Level Translation (Wu et al., Findings 2022)

ACL

• Zhaofeng Wu, Hao Peng, Nikolaos Pappas, and Noah A. Smith. 2022. Modeling Context With Linear Attention for Scalable Document-Level Translation. In Findings of the Association for Computational Linguistics: EMNLP 2022, pages 6931–6939, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics.