<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="li-chan-2019-big">
    <titleInfo>
        <title>Big Bidirectional Insertion Representations for Documents</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Lala</namePart>
        <namePart type="family">Li</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">William</namePart>
        <namePart type="family">Chan</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2019-11</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the 3rd Workshop on Neural Generation and Translation</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Alexandra</namePart>
            <namePart type="family">Birch</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Andrew</namePart>
            <namePart type="family">Finch</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Hiroaki</namePart>
            <namePart type="family">Hayashi</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Ioannis</namePart>
            <namePart type="family">Konstas</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Thang</namePart>
            <namePart type="family">Luong</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Graham</namePart>
            <namePart type="family">Neubig</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Yusuke</namePart>
            <namePart type="family">Oda</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Katsuhito</namePart>
            <namePart type="family">Sudoh</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Hong Kong</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>The Insertion Transformer is well suited for long form text generation due to its parallel generation capabilities, requiring O(łog₂ n) generation steps to generate n tokens. However, modeling long sequences is difficult, as there is more ambiguity captured in the attention mechanism. This work proposes the Big Bidirectional Insertion Representations for Documents (Big BIRD), an insertion-based model for document-level translation tasks. We scale up the insertion-based models to long form documents. Our key contribution is introducing sentence alignment via sentence-positional embeddings between the source and target document. We show an improvement of +4.3 BLEU on the WMT’19 English-\textgreaterGerman document-level translation task compared with the Insertion Transformer baseline.</abstract>
    <identifier type="citekey">li-chan-2019-big</identifier>
    <identifier type="doi">10.18653/v1/D19-5620</identifier>
    <location>
        <url>https://aclanthology.org/D19-5620</url>
    </location>
    <part>
        <date>2019-11</date>
        <extent unit="page">
            <start>194</start>
            <end>198</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Big Bidirectional Insertion Representations for Documents
%A Li, Lala
%A Chan, William
%Y Birch, Alexandra
%Y Finch, Andrew
%Y Hayashi, Hiroaki
%Y Konstas, Ioannis
%Y Luong, Thang
%Y Neubig, Graham
%Y Oda, Yusuke
%Y Sudoh, Katsuhito
%S Proceedings of the 3rd Workshop on Neural Generation and Translation
%D 2019
%8 November
%I Association for Computational Linguistics
%C Hong Kong
%F li-chan-2019-big
%X The Insertion Transformer is well suited for long form text generation due to its parallel generation capabilities, requiring O(łog₂ n) generation steps to generate n tokens. However, modeling long sequences is difficult, as there is more ambiguity captured in the attention mechanism. This work proposes the Big Bidirectional Insertion Representations for Documents (Big BIRD), an insertion-based model for document-level translation tasks. We scale up the insertion-based models to long form documents. Our key contribution is introducing sentence alignment via sentence-positional embeddings between the source and target document. We show an improvement of +4.3 BLEU on the WMT’19 English-\textgreaterGerman document-level translation task compared with the Insertion Transformer baseline.
%R 10.18653/v1/D19-5620
%U https://aclanthology.org/D19-5620
%U https://doi.org/10.18653/v1/D19-5620
%P 194-198

Markdown (Informal)

[Big Bidirectional Insertion Representations for Documents](https://aclanthology.org/D19-5620) (Li & Chan, NGT 2019)

• Big Bidirectional Insertion Representations for Documents (Li & Chan, NGT 2019)

ACL

• Lala Li and William Chan. 2019. Big Bidirectional Insertion Representations for Documents. In Proceedings of the 3rd Workshop on Neural Generation and Translation, pages 194–198, Hong Kong. Association for Computational Linguistics.