<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="wada-etal-2022-unsupervised">
    <titleInfo>
        <title>Unsupervised Lexical Substitution with Decontextualised Embeddings</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Takashi</namePart>
        <namePart type="family">Wada</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Timothy</namePart>
        <namePart type="family">Baldwin</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Yuji</namePart>
        <namePart type="family">Matsumoto</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Jey</namePart>
        <namePart type="given">Han</namePart>
        <namePart type="family">Lau</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2022-10</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the 29th International Conference on Computational Linguistics</title>
        </titleInfo>
        <originInfo>
            <publisher>International Committee on Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Gyeongju, Republic of Korea</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>We propose a new unsupervised method for lexical substitution using pre-trained language models. Compared to previous approaches that use the generative capability of language models to predict substitutes, our method retrieves substitutes based on the similarity of contextualised and decontextualised word embeddings, i.e. the average contextual representation of a word in multiple contexts. We conduct experiments in English and Italian, and show that our method substantially outperforms strong baselines and establishes a new state-of-the-art without any explicit supervision or fine-tuning. We further show that our method performs particularly well at predicting low-frequency substitutes, and also generates a diverse list of substitute candidates, reducing morphophonetic or morphosyntactic biases induced by article-noun agreement.</abstract>
    <identifier type="citekey">wada-etal-2022-unsupervised</identifier>
    <location>
        <url>https://aclanthology.org/2022.coling-1.366</url>
    </location>
    <part>
        <date>2022-10</date>
        <extent unit="page">
            <start>4172</start>
            <end>4185</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Unsupervised Lexical Substitution with Decontextualised Embeddings
%A Wada, Takashi
%A Baldwin, Timothy
%A Matsumoto, Yuji
%A Lau, Jey Han
%S Proceedings of the 29th International Conference on Computational Linguistics
%D 2022
%8 October
%I International Committee on Computational Linguistics
%C Gyeongju, Republic of Korea
%F wada-etal-2022-unsupervised
%X We propose a new unsupervised method for lexical substitution using pre-trained language models. Compared to previous approaches that use the generative capability of language models to predict substitutes, our method retrieves substitutes based on the similarity of contextualised and decontextualised word embeddings, i.e. the average contextual representation of a word in multiple contexts. We conduct experiments in English and Italian, and show that our method substantially outperforms strong baselines and establishes a new state-of-the-art without any explicit supervision or fine-tuning. We further show that our method performs particularly well at predicting low-frequency substitutes, and also generates a diverse list of substitute candidates, reducing morphophonetic or morphosyntactic biases induced by article-noun agreement.
%U https://aclanthology.org/2022.coling-1.366
%P 4172-4185

Markdown (Informal)

[Unsupervised Lexical Substitution with Decontextualised Embeddings](https://aclanthology.org/2022.coling-1.366) (Wada et al., COLING 2022)

• Unsupervised Lexical Substitution with Decontextualised Embeddings (Wada et al., COLING 2022)

ACL

• Takashi Wada, Timothy Baldwin, Yuji Matsumoto, and Jey Han Lau. 2022. Unsupervised Lexical Substitution with Decontextualised Embeddings. In Proceedings of the 29th International Conference on Computational Linguistics, pages 4172–4185, Gyeongju, Republic of Korea. International Committee on Computational Linguistics.