<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="shah-etal-2023-retrofitting">
    <titleInfo>
        <title>Retrofitting Light-weight Language Models for Emotions using Supervised Contrastive Learning</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Sapan</namePart>
        <namePart type="family">Shah</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Sreedhar</namePart>
        <namePart type="family">Reddy</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Pushpak</namePart>
        <namePart type="family">Bhattacharyya</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2023-12</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Houda</namePart>
            <namePart type="family">Bouamor</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Juan</namePart>
            <namePart type="family">Pino</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Kalika</namePart>
            <namePart type="family">Bali</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Singapore</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>We present a novel retrofitting method to induce emotion aspects into pre-trained language models (PLMs) such as BERT and RoBERTa. Our method updates pre-trained network weights using contrastive learning so that the text fragments exhibiting similar emotions are encoded nearby in the representation space, and the fragments with different emotion content are pushed apart. While doing so, it also ensures that the linguistic knowledge already present in PLMs is not inadvertently perturbed. The language models retrofitted by our method, i.e., BERTEmo and RoBERTaEmo, produce emotion-aware text representations, as evaluated through different clustering and retrieval metrics. For the downstream tasks on sentiment analysis and sarcasm detection, they perform better than their pre-trained counterparts (about 1% improvement in F1-score) and other existing approaches. Additionally, a more significant boost in performance is observed for the retrofitted models over pre-trained ones in few-shot learning setting.</abstract>
    <identifier type="citekey">shah-etal-2023-retrofitting</identifier>
    <identifier type="doi">10.18653/v1/2023.emnlp-main.222</identifier>
    <location>
        <url>https://aclanthology.org/2023.emnlp-main.222</url>
    </location>
    <part>
        <date>2023-12</date>
        <extent unit="page">
            <start>3640</start>
            <end>3654</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Retrofitting Light-weight Language Models for Emotions using Supervised Contrastive Learning
%A Shah, Sapan
%A Reddy, Sreedhar
%A Bhattacharyya, Pushpak
%Y Bouamor, Houda
%Y Pino, Juan
%Y Bali, Kalika
%S Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing
%D 2023
%8 December
%I Association for Computational Linguistics
%C Singapore
%F shah-etal-2023-retrofitting
%X We present a novel retrofitting method to induce emotion aspects into pre-trained language models (PLMs) such as BERT and RoBERTa. Our method updates pre-trained network weights using contrastive learning so that the text fragments exhibiting similar emotions are encoded nearby in the representation space, and the fragments with different emotion content are pushed apart. While doing so, it also ensures that the linguistic knowledge already present in PLMs is not inadvertently perturbed. The language models retrofitted by our method, i.e., BERTEmo and RoBERTaEmo, produce emotion-aware text representations, as evaluated through different clustering and retrieval metrics. For the downstream tasks on sentiment analysis and sarcasm detection, they perform better than their pre-trained counterparts (about 1% improvement in F1-score) and other existing approaches. Additionally, a more significant boost in performance is observed for the retrofitted models over pre-trained ones in few-shot learning setting.
%R 10.18653/v1/2023.emnlp-main.222
%U https://aclanthology.org/2023.emnlp-main.222
%U https://doi.org/10.18653/v1/2023.emnlp-main.222
%P 3640-3654

Markdown (Informal)

[Retrofitting Light-weight Language Models for Emotions using Supervised Contrastive Learning](https://aclanthology.org/2023.emnlp-main.222) (Shah et al., EMNLP 2023)

• Retrofitting Light-weight Language Models for Emotions using Supervised Contrastive Learning (Shah et al., EMNLP 2023)

ACL

• Sapan Shah, Sreedhar Reddy, and Pushpak Bhattacharyya. 2023. Retrofitting Light-weight Language Models for Emotions using Supervised Contrastive Learning. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 3640–3654, Singapore. Association for Computational Linguistics.