<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="cheng-zhang-2022-con">
    <titleInfo>
        <title>Con-NAT: Contrastive Non-autoregressive Neural Machine Translation</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Hao</namePart>
        <namePart type="family">Cheng</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Zhihua</namePart>
        <namePart type="family">Zhang</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2022-12</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Findings of the Association for Computational Linguistics: EMNLP 2022</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Yoav</namePart>
            <namePart type="family">Goldberg</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Zornitsa</namePart>
            <namePart type="family">Kozareva</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Yue</namePart>
            <namePart type="family">Zhang</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Abu Dhabi, United Arab Emirates</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>Inspired by the success of contrastive learning in natural language processing, we incorporate contrastive learning into the conditional masked language model which is extensively used in non-autoregressive neural machine translation (NAT). Accordingly, we propose a Contrastive Non-autoregressive Neural Machine Translation (Con-NAT) model. Con-NAT optimizes the similarity of several different representations of the same token in the same sentence. We propose two methods to obtain various representations: Contrastive Common Mask and Contrastive Dropout. Positive pairs are various different representations of the same token, while negative pairs are representations of different tokens. In the feature space, the model with contrastive loss pulls positive pairs together and pushes negative pairs away. We conduct extensive experiments on six translation directions with different data sizes. The results demonstrate that Con-NAT showed a consistent and significant improvement in fully and iterative NAT. Con-NAT is state-of-the-art on WMT’16 Ro-En (34.18 BLEU).</abstract>
    <identifier type="citekey">cheng-zhang-2022-con</identifier>
    <identifier type="doi">10.18653/v1/2022.findings-emnlp.463</identifier>
    <location>
        <url>https://aclanthology.org/2022.findings-emnlp.463</url>
    </location>
    <part>
        <date>2022-12</date>
        <extent unit="page">
            <start>6219</start>
            <end>6231</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Con-NAT: Contrastive Non-autoregressive Neural Machine Translation
%A Cheng, Hao
%A Zhang, Zhihua
%Y Goldberg, Yoav
%Y Kozareva, Zornitsa
%Y Zhang, Yue
%S Findings of the Association for Computational Linguistics: EMNLP 2022
%D 2022
%8 December
%I Association for Computational Linguistics
%C Abu Dhabi, United Arab Emirates
%F cheng-zhang-2022-con
%X Inspired by the success of contrastive learning in natural language processing, we incorporate contrastive learning into the conditional masked language model which is extensively used in non-autoregressive neural machine translation (NAT). Accordingly, we propose a Contrastive Non-autoregressive Neural Machine Translation (Con-NAT) model. Con-NAT optimizes the similarity of several different representations of the same token in the same sentence. We propose two methods to obtain various representations: Contrastive Common Mask and Contrastive Dropout. Positive pairs are various different representations of the same token, while negative pairs are representations of different tokens. In the feature space, the model with contrastive loss pulls positive pairs together and pushes negative pairs away. We conduct extensive experiments on six translation directions with different data sizes. The results demonstrate that Con-NAT showed a consistent and significant improvement in fully and iterative NAT. Con-NAT is state-of-the-art on WMT’16 Ro-En (34.18 BLEU).
%R 10.18653/v1/2022.findings-emnlp.463
%U https://aclanthology.org/2022.findings-emnlp.463
%U https://doi.org/10.18653/v1/2022.findings-emnlp.463
%P 6219-6231

Markdown (Informal)

[Con-NAT: Contrastive Non-autoregressive Neural Machine Translation](https://aclanthology.org/2022.findings-emnlp.463) (Cheng & Zhang, Findings 2022)

• Con-NAT: Contrastive Non-autoregressive Neural Machine Translation (Cheng & Zhang, Findings 2022)

ACL

• Hao Cheng and Zhihua Zhang. 2022. Con-NAT: Contrastive Non-autoregressive Neural Machine Translation. In Findings of the Association for Computational Linguistics: EMNLP 2022, pages 6219–6231, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics.