<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="feng-etal-2021-improving">
    <titleInfo>
        <title>Improving Character-Aware Neural Language Model by Warming up Character Encoder under Skip-gram Architecture</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Yukun</namePart>
        <namePart type="family">Feng</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Chenlong</namePart>
        <namePart type="family">Hu</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Hidetaka</namePart>
        <namePart type="family">Kamigaito</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Hiroya</namePart>
        <namePart type="family">Takamura</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Manabu</namePart>
        <namePart type="family">Okumura</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2021-09</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the International Conference on Recent Advances in Natural Language Processing (RANLP 2021)</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Ruslan</namePart>
            <namePart type="family">Mitkov</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Galia</namePart>
            <namePart type="family">Angelova</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>INCOMA Ltd.</publisher>
            <place>
                <placeTerm type="text">Held Online</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>Character-aware neural language models can capture the relationship between words by exploiting character-level information and are particularly effective for languages with rich morphology. However, these models are usually biased towards information from surface forms. To alleviate this problem, we propose a simple and effective method to improve a character-aware neural language model by forcing a character encoder to produce word-based embeddings under Skip-gram architecture in a warm-up step without extra training data. We empirically show that the resulting character-aware neural language model achieves obvious improvements of perplexity scores on typologically diverse languages, that contain many low-frequency or unseen words.</abstract>
    <identifier type="citekey">feng-etal-2021-improving</identifier>
    <location>
        <url>https://aclanthology.org/2021.ranlp-1.48</url>
    </location>
    <part>
        <date>2021-09</date>
        <extent unit="page">
            <start>421</start>
            <end>427</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Improving Character-Aware Neural Language Model by Warming up Character Encoder under Skip-gram Architecture
%A Feng, Yukun
%A Hu, Chenlong
%A Kamigaito, Hidetaka
%A Takamura, Hiroya
%A Okumura, Manabu
%Y Mitkov, Ruslan
%Y Angelova, Galia
%S Proceedings of the International Conference on Recent Advances in Natural Language Processing (RANLP 2021)
%D 2021
%8 September
%I INCOMA Ltd.
%C Held Online
%F feng-etal-2021-improving
%X Character-aware neural language models can capture the relationship between words by exploiting character-level information and are particularly effective for languages with rich morphology. However, these models are usually biased towards information from surface forms. To alleviate this problem, we propose a simple and effective method to improve a character-aware neural language model by forcing a character encoder to produce word-based embeddings under Skip-gram architecture in a warm-up step without extra training data. We empirically show that the resulting character-aware neural language model achieves obvious improvements of perplexity scores on typologically diverse languages, that contain many low-frequency or unseen words.
%U https://aclanthology.org/2021.ranlp-1.48
%P 421-427

Markdown (Informal)

[Improving Character-Aware Neural Language Model by Warming up Character Encoder under Skip-gram Architecture](https://aclanthology.org/2021.ranlp-1.48) (Feng et al., RANLP 2021)

• Improving Character-Aware Neural Language Model by Warming up Character Encoder under Skip-gram Architecture (Feng et al., RANLP 2021)

ACL

• Yukun Feng, Chenlong Hu, Hidetaka Kamigaito, Hiroya Takamura, and Manabu Okumura. 2021. Improving Character-Aware Neural Language Model by Warming up Character Encoder under Skip-gram Architecture. In Proceedings of the International Conference on Recent Advances in Natural Language Processing (RANLP 2021), pages 421–427, Held Online. INCOMA Ltd..