<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="qiu-etal-2021-contrastive">
    <titleInfo>
        <title>Contrastive Learning of Sentence Representations</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Hefei</namePart>
        <namePart type="family">Qiu</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Wei</namePart>
        <namePart type="family">Ding</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Ping</namePart>
        <namePart type="family">Chen</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2021-12</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the 18th International Conference on Natural Language Processing (ICON)</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Sivaji</namePart>
            <namePart type="family">Bandyopadhyay</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Sobha</namePart>
            <namePart type="given">Lalitha</namePart>
            <namePart type="family">Devi</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Pushpak</namePart>
            <namePart type="family">Bhattacharyya</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>NLP Association of India (NLPAI)</publisher>
            <place>
                <placeTerm type="text">National Institute of Technology Silchar, Silchar, India</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>Learning sentence representations which capture rich semantic meanings has been crucial for many NLP tasks. Pre-trained language models such as BERT have achieved great success in NLP, but sentence embeddings extracted directly from these models do not perform well without fine-tuning. We propose Contrastive Learning of Sentence Representations (CLSR), a novel approach which applies contrastive learning to learn universal sentence representations on top of pre-trained language models. CLSR utilizes semantic similarity of two sentences to construct positive instance for contrastive learning. Semantic information that has been captured by the pre-trained models is kept by getting sentence embeddings from these models with proper pooling strategy. An encoder followed by a linear projection takes these embeddings as inputs and is trained under a contrastive objective. To evaluate the performance of CLSR, we run experiments on a range of pre-trained language models and their variants on a series of Semantic Contextual Similarity tasks. Results show that CLSR gains significant performance improvements over existing SOTA language models.</abstract>
    <identifier type="citekey">qiu-etal-2021-contrastive</identifier>
    <location>
        <url>https://aclanthology.org/2021.icon-main.33</url>
    </location>
    <part>
        <date>2021-12</date>
        <extent unit="page">
            <start>277</start>
            <end>283</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Contrastive Learning of Sentence Representations
%A Qiu, Hefei
%A Ding, Wei
%A Chen, Ping
%Y Bandyopadhyay, Sivaji
%Y Devi, Sobha Lalitha
%Y Bhattacharyya, Pushpak
%S Proceedings of the 18th International Conference on Natural Language Processing (ICON)
%D 2021
%8 December
%I NLP Association of India (NLPAI)
%C National Institute of Technology Silchar, Silchar, India
%F qiu-etal-2021-contrastive
%X Learning sentence representations which capture rich semantic meanings has been crucial for many NLP tasks. Pre-trained language models such as BERT have achieved great success in NLP, but sentence embeddings extracted directly from these models do not perform well without fine-tuning. We propose Contrastive Learning of Sentence Representations (CLSR), a novel approach which applies contrastive learning to learn universal sentence representations on top of pre-trained language models. CLSR utilizes semantic similarity of two sentences to construct positive instance for contrastive learning. Semantic information that has been captured by the pre-trained models is kept by getting sentence embeddings from these models with proper pooling strategy. An encoder followed by a linear projection takes these embeddings as inputs and is trained under a contrastive objective. To evaluate the performance of CLSR, we run experiments on a range of pre-trained language models and their variants on a series of Semantic Contextual Similarity tasks. Results show that CLSR gains significant performance improvements over existing SOTA language models.
%U https://aclanthology.org/2021.icon-main.33
%P 277-283

Markdown (Informal)

[Contrastive Learning of Sentence Representations](https://aclanthology.org/2021.icon-main.33) (Qiu et al., ICON 2021)

• Contrastive Learning of Sentence Representations (Qiu et al., ICON 2021)

ACL

• Hefei Qiu, Wei Ding, and Ping Chen. 2021. Contrastive Learning of Sentence Representations. In Proceedings of the 18th International Conference on Natural Language Processing (ICON), pages 277–283, National Institute of Technology Silchar, Silchar, India. NLP Association of India (NLPAI).