<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="godin-etal-2019-learning">
    <titleInfo>
        <title>Learning When Not to Answer: a Ternary Reward Structure for Reinforcement Learning Based Question Answering</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Fréderic</namePart>
        <namePart type="family">Godin</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Anjishnu</namePart>
        <namePart type="family">Kumar</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Arpit</namePart>
        <namePart type="family">Mittal</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2019-06</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Industry Papers)</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Anastassia</namePart>
            <namePart type="family">Loukina</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Michelle</namePart>
            <namePart type="family">Morales</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Rohit</namePart>
            <namePart type="family">Kumar</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Minneapolis, Minnesota</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>In this paper, we investigate the challenges of using reinforcement learning agents for question-answering over knowledge graphs for real-world applications. We examine the performance metrics used by state-of-the-art systems and determine that they are inadequate for such settings. More specifically, they do not evaluate the systems correctly for situations when there is no answer available and thus agents optimized for these metrics are poor at modeling confidence. We introduce a simple new performance metric for evaluating question-answering agents that is more representative of practical usage conditions, and optimize for this metric by extending the binary reward structure used in prior work to a ternary reward structure which also rewards an agent for not answering a question rather than giving an incorrect answer. We show that this can drastically improve the precision of answered questions while only not answering a limited number of previously correctly answered questions. Employing a supervised learning strategy using depth-first-search paths to bootstrap the reinforcement learning algorithm further improves performance.</abstract>
    <identifier type="citekey">godin-etal-2019-learning</identifier>
    <identifier type="doi">10.18653/v1/N19-2016</identifier>
    <location>
        <url>https://aclanthology.org/N19-2016</url>
    </location>
    <part>
        <date>2019-06</date>
        <extent unit="page">
            <start>122</start>
            <end>129</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Learning When Not to Answer: a Ternary Reward Structure for Reinforcement Learning Based Question Answering
%A Godin, Fréderic
%A Kumar, Anjishnu
%A Mittal, Arpit
%Y Loukina, Anastassia
%Y Morales, Michelle
%Y Kumar, Rohit
%S Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Industry Papers)
%D 2019
%8 June
%I Association for Computational Linguistics
%C Minneapolis, Minnesota
%F godin-etal-2019-learning
%X In this paper, we investigate the challenges of using reinforcement learning agents for question-answering over knowledge graphs for real-world applications. We examine the performance metrics used by state-of-the-art systems and determine that they are inadequate for such settings. More specifically, they do not evaluate the systems correctly for situations when there is no answer available and thus agents optimized for these metrics are poor at modeling confidence. We introduce a simple new performance metric for evaluating question-answering agents that is more representative of practical usage conditions, and optimize for this metric by extending the binary reward structure used in prior work to a ternary reward structure which also rewards an agent for not answering a question rather than giving an incorrect answer. We show that this can drastically improve the precision of answered questions while only not answering a limited number of previously correctly answered questions. Employing a supervised learning strategy using depth-first-search paths to bootstrap the reinforcement learning algorithm further improves performance.
%R 10.18653/v1/N19-2016
%U https://aclanthology.org/N19-2016
%U https://doi.org/10.18653/v1/N19-2016
%P 122-129

Markdown (Informal)

[Learning When Not to Answer: a Ternary Reward Structure for Reinforcement Learning Based Question Answering](https://aclanthology.org/N19-2016) (Godin et al., NAACL 2019)

• Learning When Not to Answer: a Ternary Reward Structure for Reinforcement Learning Based Question Answering (Godin et al., NAACL 2019)

ACL

• Fréderic Godin, Anjishnu Kumar, and Arpit Mittal. 2019. Learning When Not to Answer: a Ternary Reward Structure for Reinforcement Learning Based Question Answering. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Industry Papers), pages 122–129, Minneapolis, Minnesota. Association for Computational Linguistics.