<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="keung-etal-2020-unsupervised">
    <titleInfo>
        <title>Unsupervised Bitext Mining and Translation via Self-Trained Contextual Embeddings</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Phillip</namePart>
        <namePart type="family">Keung</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Julian</namePart>
        <namePart type="family">Salazar</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Yichao</namePart>
        <namePart type="family">Lu</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Noah</namePart>
        <namePart type="given">A</namePart>
        <namePart type="family">Smith</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2020</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <genre authority="bibutilsgt">journal article</genre>
    <relatedItem type="host">
        <titleInfo>
            <title>Transactions of the Association for Computational Linguistics</title>
        </titleInfo>
        <originInfo>
            <issuance>continuing</issuance>
            <publisher>MIT Press</publisher>
            <place>
                <placeTerm type="text">Cambridge, MA</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">periodical</genre>
        <genre authority="bibutilsgt">academic journal</genre>
    </relatedItem>
    <abstract>We describe an unsupervised method to create pseudo-parallel corpora for machine translation (MT) from unaligned text. We use multilingual BERT to create source and target sentence embeddings for nearest-neighbor search and adapt the model via self-training. We validate our technique by extracting parallel sentence pairs on the BUCC 2017 bitext mining task and observe up to a 24.5 point increase (absolute) in F1 scores over previous unsupervised methods. We then improve an XLM-based unsupervised neural MT system pre-trained on Wikipedia by supplementing it with pseudo-parallel text mined from the same corpus, boosting unsupervised translation performance by up to 3.5 BLEU on the WMT‘14 French-English and WMT‘16 German-English tasks and outperforming the previous state-of-the-art. Finally, we enrich the IWSLT‘15 English-Vietnamese corpus with pseudo-parallel Wikipedia sentence pairs, yielding a 1.2 BLEU improvement on the low-resource MT task. We demonstrate that unsupervised bitext mining is an effective way of augmenting MT datasets and complements existing techniques like initializing with pre-trained contextual embeddings.</abstract>
    <identifier type="citekey">keung-etal-2020-unsupervised</identifier>
    <identifier type="doi">10.1162/tacl_a_00348</identifier>
    <location>
        <url>https://aclanthology.org/2020.tacl-1.53/</url>
    </location>
    <part>
        <date>2020</date>
        <detail type="volume"><number>8</number></detail>
        <extent unit="page">
            <start>828</start>
            <end>841</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Journal Article
%T Unsupervised Bitext Mining and Translation via Self-Trained Contextual Embeddings
%A Keung, Phillip
%A Salazar, Julian
%A Lu, Yichao
%A Smith, Noah A.
%J Transactions of the Association for Computational Linguistics
%D 2020
%V 8
%I MIT Press
%C Cambridge, MA
%F keung-etal-2020-unsupervised
%X We describe an unsupervised method to create pseudo-parallel corpora for machine translation (MT) from unaligned text. We use multilingual BERT to create source and target sentence embeddings for nearest-neighbor search and adapt the model via self-training. We validate our technique by extracting parallel sentence pairs on the BUCC 2017 bitext mining task and observe up to a 24.5 point increase (absolute) in F1 scores over previous unsupervised methods. We then improve an XLM-based unsupervised neural MT system pre-trained on Wikipedia by supplementing it with pseudo-parallel text mined from the same corpus, boosting unsupervised translation performance by up to 3.5 BLEU on the WMT‘14 French-English and WMT‘16 German-English tasks and outperforming the previous state-of-the-art. Finally, we enrich the IWSLT‘15 English-Vietnamese corpus with pseudo-parallel Wikipedia sentence pairs, yielding a 1.2 BLEU improvement on the low-resource MT task. We demonstrate that unsupervised bitext mining is an effective way of augmenting MT datasets and complements existing techniques like initializing with pre-trained contextual embeddings.
%R 10.1162/tacl_a_00348
%U https://aclanthology.org/2020.tacl-1.53/
%U https://doi.org/10.1162/tacl_a_00348
%P 828-841

Markdown (Informal)

[Unsupervised Bitext Mining and Translation via Self-Trained Contextual Embeddings](https://aclanthology.org/2020.tacl-1.53/) (Keung et al., TACL 2020)

• Unsupervised Bitext Mining and Translation via Self-Trained Contextual Embeddings (Keung et al., TACL 2020)

ACL

• Phillip Keung, Julian Salazar, Yichao Lu, and Noah A. Smith. 2020. Unsupervised Bitext Mining and Translation via Self-Trained Contextual Embeddings. Transactions of the Association for Computational Linguistics, 8:828–841.