<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="wu-etal-2016-empirical">
    <titleInfo>
        <title>An Empirical Exploration of Skip Connections for Sequential Tagging</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Huijia</namePart>
        <namePart type="family">Wu</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Jiajun</namePart>
        <namePart type="family">Zhang</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Chengqing</namePart>
        <namePart type="family">Zong</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2016-12</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Yuji</namePart>
            <namePart type="family">Matsumoto</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Rashmi</namePart>
            <namePart type="family">Prasad</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>The COLING 2016 Organizing Committee</publisher>
            <place>
                <placeTerm type="text">Osaka, Japan</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>In this paper, we empirically explore the effects of various kinds of skip connections in stacked bidirectional LSTMs for sequential tagging. We investigate three kinds of skip connections connecting to LSTM cells: (a) skip connections to the gates, (b) skip connections to the internal states and (c) skip connections to the cell outputs. We present comprehensive experiments showing that skip connections to cell outputs outperform the remaining two. Furthermore, we observe that using gated identity functions as skip mappings works pretty well. Based on this novel skip connections, we successfully train deep stacked bidirectional LSTM models and obtain state-of-the-art results on CCG supertagging and comparable results on POS tagging.</abstract>
    <identifier type="citekey">wu-etal-2016-empirical</identifier>
    <location>
        <url>https://aclanthology.org/C16-1020</url>
    </location>
    <part>
        <date>2016-12</date>
        <extent unit="page">
            <start>203</start>
            <end>212</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T An Empirical Exploration of Skip Connections for Sequential Tagging
%A Wu, Huijia
%A Zhang, Jiajun
%A Zong, Chengqing
%Y Matsumoto, Yuji
%Y Prasad, Rashmi
%S Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers
%D 2016
%8 December
%I The COLING 2016 Organizing Committee
%C Osaka, Japan
%F wu-etal-2016-empirical
%X In this paper, we empirically explore the effects of various kinds of skip connections in stacked bidirectional LSTMs for sequential tagging. We investigate three kinds of skip connections connecting to LSTM cells: (a) skip connections to the gates, (b) skip connections to the internal states and (c) skip connections to the cell outputs. We present comprehensive experiments showing that skip connections to cell outputs outperform the remaining two. Furthermore, we observe that using gated identity functions as skip mappings works pretty well. Based on this novel skip connections, we successfully train deep stacked bidirectional LSTM models and obtain state-of-the-art results on CCG supertagging and comparable results on POS tagging.
%U https://aclanthology.org/C16-1020
%P 203-212

Markdown (Informal)

[An Empirical Exploration of Skip Connections for Sequential Tagging](https://aclanthology.org/C16-1020) (Wu et al., COLING 2016)

• An Empirical Exploration of Skip Connections for Sequential Tagging (Wu et al., COLING 2016)

ACL

• Huijia Wu, Jiajun Zhang, and Chengqing Zong. 2016. An Empirical Exploration of Skip Connections for Sequential Tagging. In Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers, pages 203–212, Osaka, Japan. The COLING 2016 Organizing Committee.