<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="chen-etal-2022-imputing">
    <titleInfo>
        <title>Imputing Out-of-Vocabulary Embeddings with LOVE Makes LanguageModels Robust with Little Cost</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Lihu</namePart>
        <namePart type="family">Chen</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Gael</namePart>
        <namePart type="family">Varoquaux</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Fabian</namePart>
        <namePart type="family">Suchanek</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2022-05</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Smaranda</namePart>
            <namePart type="family">Muresan</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Preslav</namePart>
            <namePart type="family">Nakov</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Aline</namePart>
            <namePart type="family">Villavicencio</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Dublin, Ireland</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>State-of-the-art NLP systems represent inputs with word embeddings, but these are brittle when faced with Out-of-Vocabulary (OOV) words. To address this issue, we follow the principle of mimick-like models to generate vectors for unseen words, by learning the behavior of pre-trained embeddings using only the surface form of words. We present a simple contrastive learning framework, LOVE, which extends the word representation of an existing pre-trained language model (such as BERT) and makes it robust to OOV with few additional parameters. Extensive evaluations demonstrate that our lightweight model achieves similar or even better performances than prior competitors, both on original datasets and on corrupted variants. Moreover, it can be used in a plug-and-play fashion with FastText and BERT, where it significantly improves their robustness.</abstract>
    <identifier type="citekey">chen-etal-2022-imputing</identifier>
    <identifier type="doi">10.18653/v1/2022.acl-long.245</identifier>
    <location>
        <url>https://aclanthology.org/2022.acl-long.245</url>
    </location>
    <part>
        <date>2022-05</date>
        <extent unit="page">
            <start>3488</start>
            <end>3504</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Imputing Out-of-Vocabulary Embeddings with LOVE Makes LanguageModels Robust with Little Cost
%A Chen, Lihu
%A Varoquaux, Gael
%A Suchanek, Fabian
%Y Muresan, Smaranda
%Y Nakov, Preslav
%Y Villavicencio, Aline
%S Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
%D 2022
%8 May
%I Association for Computational Linguistics
%C Dublin, Ireland
%F chen-etal-2022-imputing
%X State-of-the-art NLP systems represent inputs with word embeddings, but these are brittle when faced with Out-of-Vocabulary (OOV) words. To address this issue, we follow the principle of mimick-like models to generate vectors for unseen words, by learning the behavior of pre-trained embeddings using only the surface form of words. We present a simple contrastive learning framework, LOVE, which extends the word representation of an existing pre-trained language model (such as BERT) and makes it robust to OOV with few additional parameters. Extensive evaluations demonstrate that our lightweight model achieves similar or even better performances than prior competitors, both on original datasets and on corrupted variants. Moreover, it can be used in a plug-and-play fashion with FastText and BERT, where it significantly improves their robustness.
%R 10.18653/v1/2022.acl-long.245
%U https://aclanthology.org/2022.acl-long.245
%U https://doi.org/10.18653/v1/2022.acl-long.245
%P 3488-3504

Markdown (Informal)

[Imputing Out-of-Vocabulary Embeddings with LOVE Makes LanguageModels Robust with Little Cost](https://aclanthology.org/2022.acl-long.245) (Chen et al., ACL 2022)

• Imputing Out-of-Vocabulary Embeddings with LOVE Makes LanguageModels Robust with Little Cost (Chen et al., ACL 2022)

ACL

• Lihu Chen, Gael Varoquaux, and Fabian Suchanek. 2022. Imputing Out-of-Vocabulary Embeddings with LOVE Makes LanguageModels Robust with Little Cost. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 3488–3504, Dublin, Ireland. Association for Computational Linguistics.