<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="arora-etal-2020-hint3">
    <titleInfo>
        <title>HINT3: Raising the bar for Intent Detection in the Wild</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Gaurav</namePart>
        <namePart type="family">Arora</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Chirag</namePart>
        <namePart type="family">Jain</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Manas</namePart>
        <namePart type="family">Chaturvedi</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Krupal</namePart>
        <namePart type="family">Modi</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2020-11</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the First Workshop on Insights from Negative Results in NLP</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Anna</namePart>
            <namePart type="family">Rogers</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">João</namePart>
            <namePart type="family">Sedoc</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Anna</namePart>
            <namePart type="family">Rumshisky</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Online</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>Intent Detection systems in the real world are exposed to complexities of imbalanced datasets containing varying perception of intent, unintended correlations and domain-specific aberrations. To facilitate benchmarking which can reflect near real-world scenarios, we introduce 3 new datasets created from live chatbots in diverse domains. Unlike most existing datasets that are crowdsourced, our datasets contain real user queries received by the chatbots and facilitates penalising unwanted correlations grasped during the training process. We evaluate 4 NLU platforms and a BERT based classifier and find that performance saturates at inadequate levels on test sets because all systems latch on to unintended patterns in training data.</abstract>
    <identifier type="citekey">arora-etal-2020-hint3</identifier>
    <identifier type="doi">10.18653/v1/2020.insights-1.16</identifier>
    <location>
        <url>https://aclanthology.org/2020.insights-1.16</url>
    </location>
    <part>
        <date>2020-11</date>
        <extent unit="page">
            <start>100</start>
            <end>105</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T HINT3: Raising the bar for Intent Detection in the Wild
%A Arora, Gaurav
%A Jain, Chirag
%A Chaturvedi, Manas
%A Modi, Krupal
%Y Rogers, Anna
%Y Sedoc, João
%Y Rumshisky, Anna
%S Proceedings of the First Workshop on Insights from Negative Results in NLP
%D 2020
%8 November
%I Association for Computational Linguistics
%C Online
%F arora-etal-2020-hint3
%X Intent Detection systems in the real world are exposed to complexities of imbalanced datasets containing varying perception of intent, unintended correlations and domain-specific aberrations. To facilitate benchmarking which can reflect near real-world scenarios, we introduce 3 new datasets created from live chatbots in diverse domains. Unlike most existing datasets that are crowdsourced, our datasets contain real user queries received by the chatbots and facilitates penalising unwanted correlations grasped during the training process. We evaluate 4 NLU platforms and a BERT based classifier and find that performance saturates at inadequate levels on test sets because all systems latch on to unintended patterns in training data.
%R 10.18653/v1/2020.insights-1.16
%U https://aclanthology.org/2020.insights-1.16
%U https://doi.org/10.18653/v1/2020.insights-1.16
%P 100-105

Markdown (Informal)

[HINT3: Raising the bar for Intent Detection in the Wild](https://aclanthology.org/2020.insights-1.16) (Arora et al., insights 2020)

• HINT3: Raising the bar for Intent Detection in the Wild (Arora et al., insights 2020)

ACL

• Gaurav Arora, Chirag Jain, Manas Chaturvedi, and Krupal Modi. 2020. HINT3: Raising the bar for Intent Detection in the Wild. In Proceedings of the First Workshop on Insights from Negative Results in NLP, pages 100–105, Online. Association for Computational Linguistics.