<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="lee-etal-2022-normalizing">
    <titleInfo>
        <title>Normalizing Mutual Information for Robust Adaptive Training for Translation</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Youngwon</namePart>
        <namePart type="family">Lee</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Changmin</namePart>
        <namePart type="family">Lee</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Hojin</namePart>
        <namePart type="family">Lee</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Seung-won</namePart>
        <namePart type="family">Hwang</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2022-12</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Yoav</namePart>
            <namePart type="family">Goldberg</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Zornitsa</namePart>
            <namePart type="family">Kozareva</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Yue</namePart>
            <namePart type="family">Zhang</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Abu Dhabi, United Arab Emirates</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>Despite the success of neural machine translation models, tensions between fluency of optimizing target language modeling and source-faithfulness remain as challenges. Previously, Conditional Bilingual Mutual Information (CBMI), a scoring metric for the importance of target sentences and tokens, was proposed to encourage fluent and faithful translations. The score is obtained by combining the probability from the translation model and the target language model, which is then used to assign different weights to losses from sentences and tokens. Meanwhile, we argue this metric is not properly normalized, for which we propose Normalized Pointwise Mutual Information (NPMI). NPMI utilizes an additional language model on source language to approximate the joint likelihood of source-target pair and the likelihood of the source, which is then used for normalizing the score. We showed that NPMI better captures the dependence between source-target and that NPMI-based token-level adaptive training brings improvements over baselines with empirical results from En-De, De-En, and En-Ro translation tasks.</abstract>
    <identifier type="citekey">lee-etal-2022-normalizing</identifier>
    <identifier type="doi">10.18653/v1/2022.emnlp-main.547</identifier>
    <location>
        <url>https://aclanthology.org/2022.emnlp-main.547</url>
    </location>
    <part>
        <date>2022-12</date>
        <extent unit="page">
            <start>8008</start>
            <end>8015</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Normalizing Mutual Information for Robust Adaptive Training for Translation
%A Lee, Youngwon
%A Lee, Changmin
%A Lee, Hojin
%A Hwang, Seung-won
%Y Goldberg, Yoav
%Y Kozareva, Zornitsa
%Y Zhang, Yue
%S Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing
%D 2022
%8 December
%I Association for Computational Linguistics
%C Abu Dhabi, United Arab Emirates
%F lee-etal-2022-normalizing
%X Despite the success of neural machine translation models, tensions between fluency of optimizing target language modeling and source-faithfulness remain as challenges. Previously, Conditional Bilingual Mutual Information (CBMI), a scoring metric for the importance of target sentences and tokens, was proposed to encourage fluent and faithful translations. The score is obtained by combining the probability from the translation model and the target language model, which is then used to assign different weights to losses from sentences and tokens. Meanwhile, we argue this metric is not properly normalized, for which we propose Normalized Pointwise Mutual Information (NPMI). NPMI utilizes an additional language model on source language to approximate the joint likelihood of source-target pair and the likelihood of the source, which is then used for normalizing the score. We showed that NPMI better captures the dependence between source-target and that NPMI-based token-level adaptive training brings improvements over baselines with empirical results from En-De, De-En, and En-Ro translation tasks.
%R 10.18653/v1/2022.emnlp-main.547
%U https://aclanthology.org/2022.emnlp-main.547
%U https://doi.org/10.18653/v1/2022.emnlp-main.547
%P 8008-8015

Markdown (Informal)

[Normalizing Mutual Information for Robust Adaptive Training for Translation](https://aclanthology.org/2022.emnlp-main.547) (Lee et al., EMNLP 2022)

• Normalizing Mutual Information for Robust Adaptive Training for Translation (Lee et al., EMNLP 2022)

ACL

• Youngwon Lee, Changmin Lee, Hojin Lee, and Seung-won Hwang. 2022. Normalizing Mutual Information for Robust Adaptive Training for Translation. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 8008–8015, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics.