<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="saha-etal-2022-similarity">
    <titleInfo>
        <title>Similarity Based Label Smoothing For Dialogue Generation</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Sougata</namePart>
        <namePart type="family">Saha</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Souvik</namePart>
        <namePart type="family">Das</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Rohini</namePart>
        <namePart type="family">Srihari</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2022-12</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the 19th International Conference on Natural Language Processing (ICON)</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Md.</namePart>
            <namePart type="given">Shad</namePart>
            <namePart type="family">Akhtar</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Tanmoy</namePart>
            <namePart type="family">Chakraborty</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">New Delhi, India</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>Generative neural conversational systems are typically trained by minimizing the entropy loss between the training “hard” targets and the predicted logits. Performance gains and improved generalization are often achieved by employing regularization techniques like label smoothing, which converts the training “hard” targets to soft targets. However, label smoothing enforces a data independent uniform distribution on the incorrect training targets, leading to a false assumption of equiprobability. In this paper, we propose and experiment with incorporating data-dependent word similarity-based weighing methods to transform the uniform distribution of the incorrect target probabilities in label smoothing to a more realistic distribution based on semantics. We introduce hyperparameters to control the incorrect target distribution and report significant performance gains over networks trained using standard label smoothing-based loss on two standard open-domain dialogue corpora.</abstract>
    <identifier type="citekey">saha-etal-2022-similarity</identifier>
    <location>
        <url>https://aclanthology.org/2022.icon-main.31</url>
    </location>
    <part>
        <date>2022-12</date>
        <extent unit="page">
            <start>253</start>
            <end>259</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Similarity Based Label Smoothing For Dialogue Generation
%A Saha, Sougata
%A Das, Souvik
%A Srihari, Rohini
%Y Akhtar, Md. Shad
%Y Chakraborty, Tanmoy
%S Proceedings of the 19th International Conference on Natural Language Processing (ICON)
%D 2022
%8 December
%I Association for Computational Linguistics
%C New Delhi, India
%F saha-etal-2022-similarity
%X Generative neural conversational systems are typically trained by minimizing the entropy loss between the training “hard” targets and the predicted logits. Performance gains and improved generalization are often achieved by employing regularization techniques like label smoothing, which converts the training “hard” targets to soft targets. However, label smoothing enforces a data independent uniform distribution on the incorrect training targets, leading to a false assumption of equiprobability. In this paper, we propose and experiment with incorporating data-dependent word similarity-based weighing methods to transform the uniform distribution of the incorrect target probabilities in label smoothing to a more realistic distribution based on semantics. We introduce hyperparameters to control the incorrect target distribution and report significant performance gains over networks trained using standard label smoothing-based loss on two standard open-domain dialogue corpora.
%U https://aclanthology.org/2022.icon-main.31
%P 253-259

Markdown (Informal)

[Similarity Based Label Smoothing For Dialogue Generation](https://aclanthology.org/2022.icon-main.31) (Saha et al., ICON 2022)

• Similarity Based Label Smoothing For Dialogue Generation (Saha et al., ICON 2022)

ACL

• Sougata Saha, Souvik Das, and Rohini Srihari. 2022. Similarity Based Label Smoothing For Dialogue Generation. In Proceedings of the 19th International Conference on Natural Language Processing (ICON), pages 253–259, New Delhi, India. Association for Computational Linguistics.