<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="lawrence-etal-2017-counterfactual">
    <titleInfo>
        <title>Counterfactual Learning from Bandit Feedback under Deterministic Logging : A Case Study in Statistical Machine Translation</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Carolin</namePart>
        <namePart type="family">Lawrence</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Artem</namePart>
        <namePart type="family">Sokolov</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Stefan</namePart>
        <namePart type="family">Riezler</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2017-09</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Martha</namePart>
            <namePart type="family">Palmer</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Rebecca</namePart>
            <namePart type="family">Hwa</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Sebastian</namePart>
            <namePart type="family">Riedel</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Copenhagen, Denmark</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>The goal of counterfactual learning for statistical machine translation (SMT) is to optimize a target SMT system from logged data that consist of user feedback to translations that were predicted by another, historic SMT system. A challenge arises by the fact that risk-averse commercial SMT systems deterministically log the most probable translation. The lack of sufficient exploration of the SMT output space seemingly contradicts the theoretical requirements for counterfactual learning. We show that counterfactual learning from deterministic bandit logs is possible nevertheless by smoothing out deterministic components in learning. This can be achieved by additive and multiplicative control variates that avoid degenerate behavior in empirical risk minimization. Our simulation experiments show improvements of up to 2 BLEU points by counterfactual learning from deterministic bandit feedback.</abstract>
    <identifier type="citekey">lawrence-etal-2017-counterfactual</identifier>
    <identifier type="doi">10.18653/v1/D17-1272</identifier>
    <location>
        <url>https://aclanthology.org/D17-1272</url>
    </location>
    <part>
        <date>2017-09</date>
        <extent unit="page">
            <start>2566</start>
            <end>2576</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Counterfactual Learning from Bandit Feedback under Deterministic Logging : A Case Study in Statistical Machine Translation
%A Lawrence, Carolin
%A Sokolov, Artem
%A Riezler, Stefan
%Y Palmer, Martha
%Y Hwa, Rebecca
%Y Riedel, Sebastian
%S Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing
%D 2017
%8 September
%I Association for Computational Linguistics
%C Copenhagen, Denmark
%F lawrence-etal-2017-counterfactual
%X The goal of counterfactual learning for statistical machine translation (SMT) is to optimize a target SMT system from logged data that consist of user feedback to translations that were predicted by another, historic SMT system. A challenge arises by the fact that risk-averse commercial SMT systems deterministically log the most probable translation. The lack of sufficient exploration of the SMT output space seemingly contradicts the theoretical requirements for counterfactual learning. We show that counterfactual learning from deterministic bandit logs is possible nevertheless by smoothing out deterministic components in learning. This can be achieved by additive and multiplicative control variates that avoid degenerate behavior in empirical risk minimization. Our simulation experiments show improvements of up to 2 BLEU points by counterfactual learning from deterministic bandit feedback.
%R 10.18653/v1/D17-1272
%U https://aclanthology.org/D17-1272
%U https://doi.org/10.18653/v1/D17-1272
%P 2566-2576

Markdown (Informal)

[Counterfactual Learning from Bandit Feedback under Deterministic Logging : A Case Study in Statistical Machine Translation](https://aclanthology.org/D17-1272) (Lawrence et al., EMNLP 2017)

• Counterfactual Learning from Bandit Feedback under Deterministic Logging : A Case Study in Statistical Machine Translation (Lawrence et al., EMNLP 2017)

ACL

• Carolin Lawrence, Artem Sokolov, and Stefan Riezler. 2017. Counterfactual Learning from Bandit Feedback under Deterministic Logging : A Case Study in Statistical Machine Translation. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 2566–2576, Copenhagen, Denmark. Association for Computational Linguistics.