Learning to Embed Words in Context for Syntactic Tasks

Lifu Tu; Kevin Gimpel; Karen Livescu

doi:10.18653/v1/W17-2632

Learning to Embed Words in Context for Syntactic Tasks

Correct Metadata for

Important: The Anthology treat PDFs as authoritative. Please use this form only to correct data that is out of line with the PDF. See our corrections guidelines if you need to change the PDF.

Title Adjust the title. Retain tags such as <fixed-case>.

Authors Adjust author names and order to match the PDF.

Abstract Correct abstract if needed. Retain XML formatting tags such as <tex-math>.

Verification against PDF Ensure that the new title/authors match the snapshot below. (If there is no snapshot or it is too small, consult the PDF.)

Authors concatenated from the text boxes above:

ALL author names match the snapshot above—including middle initials, hyphens, and accents.

Abstract

We present models for embedding words in the context of surrounding words. Such models, which we refer to as token embeddings, represent the characteristics of a word that are specific to a given context, such as word sense, syntactic category, and semantic role. We explore simple, efficient token embedding models based on standard neural network architectures. We learn token embeddings on a large amount of unannotated text and evaluate them as features for part-of-speech taggers and dependency parsers trained on much smaller amounts of annotated data. We find that predictors endowed with token embeddings consistently outperform baseline predictors across a range of context window and training set sizes.

Anthology ID:: W17-2632
Volume:: Proceedings of the 2nd Workshop on Representation Learning for NLP
Month:: August
Year:: 2017
Address:: Vancouver, Canada
Editors:: Phil Blunsom, Antoine Bordes, Kyunghyun Cho, Shay Cohen, Chris Dyer, Edward Grefenstette, Karl Moritz Hermann, Laura Rimell, Jason Weston, Scott Yih
Venue:: RepL4NLP
SIG:: SIGREP
Publisher:: Association for Computational Linguistics
Note:
Pages:: 265–275
Language:
URL:: https://aclanthology.org/W17-2632/
DOI:: 10.18653/v1/W17-2632
Bibkey:
Cite (ACL):: Lifu Tu, Kevin Gimpel, and Karen Livescu. 2017. Learning to Embed Words in Context for Syntactic Tasks. In Proceedings of the 2nd Workshop on Representation Learning for NLP, pages 265–275, Vancouver, Canada. Association for Computational Linguistics.
Cite (Informal):: Learning to Embed Words in Context for Syntactic Tasks (Tu et al., RepL4NLP 2017)
Copy Citation:
PDF:: https://aclanthology.org/W17-2632.pdf

PDF Cite Search Fix data

Export citation

BibTeX
MODS XML
Endnote
Preformatted

@inproceedings{tu-etal-2017-learning,
    title = "Learning to Embed Words in Context for Syntactic Tasks",
    author = "Tu, Lifu  and
      Gimpel, Kevin  and
      Livescu, Karen",
    editor = "Blunsom, Phil  and
      Bordes, Antoine  and
      Cho, Kyunghyun  and
      Cohen, Shay  and
      Dyer, Chris  and
      Grefenstette, Edward  and
      Hermann, Karl Moritz  and
      Rimell, Laura  and
      Weston, Jason  and
      Yih, Scott",
    booktitle = "Proceedings of the 2nd Workshop on Representation Learning for {NLP}",
    month = aug,
    year = "2017",
    address = "Vancouver, Canada",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/W17-2632/",
    doi = "10.18653/v1/W17-2632",
    pages = "265--275",
    abstract = "We present models for embedding words in the context of surrounding words. Such models, which we refer to as token embeddings, represent the characteristics of a word that are specific to a given context, such as word sense, syntactic category, and semantic role. We explore simple, efficient token embedding models based on standard neural network architectures. We learn token embeddings on a large amount of unannotated text and evaluate them as features for part-of-speech taggers and dependency parsers trained on much smaller amounts of annotated data. We find that predictors endowed with token embeddings consistently outperform baseline predictors across a range of context window and training set sizes."
}

Download as File

<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="tu-etal-2017-learning">
    <titleInfo>
        <title>Learning to Embed Words in Context for Syntactic Tasks</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Lifu</namePart>
        <namePart type="family">Tu</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Kevin</namePart>
        <namePart type="family">Gimpel</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Karen</namePart>
        <namePart type="family">Livescu</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2017-08</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the 2nd Workshop on Representation Learning for NLP</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Phil</namePart>
            <namePart type="family">Blunsom</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Antoine</namePart>
            <namePart type="family">Bordes</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Kyunghyun</namePart>
            <namePart type="family">Cho</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Shay</namePart>
            <namePart type="family">Cohen</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Chris</namePart>
            <namePart type="family">Dyer</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Edward</namePart>
            <namePart type="family">Grefenstette</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Karl</namePart>
            <namePart type="given">Moritz</namePart>
            <namePart type="family">Hermann</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Laura</namePart>
            <namePart type="family">Rimell</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Jason</namePart>
            <namePart type="family">Weston</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Scott</namePart>
            <namePart type="family">Yih</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Vancouver, Canada</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>We present models for embedding words in the context of surrounding words. Such models, which we refer to as token embeddings, represent the characteristics of a word that are specific to a given context, such as word sense, syntactic category, and semantic role. We explore simple, efficient token embedding models based on standard neural network architectures. We learn token embeddings on a large amount of unannotated text and evaluate them as features for part-of-speech taggers and dependency parsers trained on much smaller amounts of annotated data. We find that predictors endowed with token embeddings consistently outperform baseline predictors across a range of context window and training set sizes.</abstract>
    <identifier type="citekey">tu-etal-2017-learning</identifier>
    <identifier type="doi">10.18653/v1/W17-2632</identifier>
    <location>
        <url>https://aclanthology.org/W17-2632/</url>
    </location>
    <part>
        <date>2017-08</date>
        <extent unit="page">
            <start>265</start>
            <end>275</end>
        </extent>
    </part>
</mods>
</modsCollection>

Download as File

%0 Conference Proceedings
%T Learning to Embed Words in Context for Syntactic Tasks
%A Tu, Lifu
%A Gimpel, Kevin
%A Livescu, Karen
%Y Blunsom, Phil
%Y Bordes, Antoine
%Y Cho, Kyunghyun
%Y Cohen, Shay
%Y Dyer, Chris
%Y Grefenstette, Edward
%Y Hermann, Karl Moritz
%Y Rimell, Laura
%Y Weston, Jason
%Y Yih, Scott
%S Proceedings of the 2nd Workshop on Representation Learning for NLP
%D 2017
%8 August
%I Association for Computational Linguistics
%C Vancouver, Canada
%F tu-etal-2017-learning
%X We present models for embedding words in the context of surrounding words. Such models, which we refer to as token embeddings, represent the characteristics of a word that are specific to a given context, such as word sense, syntactic category, and semantic role. We explore simple, efficient token embedding models based on standard neural network architectures. We learn token embeddings on a large amount of unannotated text and evaluate them as features for part-of-speech taggers and dependency parsers trained on much smaller amounts of annotated data. We find that predictors endowed with token embeddings consistently outperform baseline predictors across a range of context window and training set sizes.
%R 10.18653/v1/W17-2632
%U https://aclanthology.org/W17-2632/
%U https://doi.org/10.18653/v1/W17-2632
%P 265-275

Download as File

Markdown (Informal)

[Learning to Embed Words in Context for Syntactic Tasks](https://aclanthology.org/W17-2632/) (Tu et al., RepL4NLP 2017)

Learning to Embed Words in Context for Syntactic Tasks (Tu et al., RepL4NLP 2017)

ACL

Lifu Tu, Kevin Gimpel, and Karen Livescu. 2017. Learning to Embed Words in Context for Syntactic Tasks. In Proceedings of the 2nd Workshop on Representation Learning for NLP, pages 265–275, Vancouver, Canada. Association for Computational Linguistics.