<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="tong-etal-2019-improving">
    <titleInfo>
        <title>Improving Natural Language Understanding by Reverse Mapping Bytepair Encoding</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Chaodong</namePart>
        <namePart type="family">Tong</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Huailiang</namePart>
        <namePart type="family">Peng</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Qiong</namePart>
        <namePart type="family">Dai</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Lei</namePart>
        <namePart type="family">Jiang</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Jianghua</namePart>
        <namePart type="family">Huang</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2019-11</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the 23rd Conference on Computational Natural Language Learning (CoNLL)</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Mohit</namePart>
            <namePart type="family">Bansal</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Aline</namePart>
            <namePart type="family">Villavicencio</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Hong Kong, China</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>We propose a method called reverse mapping bytepair encoding, which maps named-entity information and other word-level linguistic features back to subwords during the encoding procedure of bytepair encoding (BPE). We employ this method to the Generative Pre-trained Transformer (OpenAI GPT) by adding a weighted linear layer after the embedding layer. We also propose a new model architecture named as the multi-channel separate transformer to employ a training process without parameter-sharing. Evaluation on Stories Cloze, RTE, SciTail and SST-2 datasets demonstrates the effectiveness of our approach.</abstract>
    <identifier type="citekey">tong-etal-2019-improving</identifier>
    <identifier type="doi">10.18653/v1/K19-1016</identifier>
    <location>
        <url>https://aclanthology.org/K19-1016</url>
    </location>
    <part>
        <date>2019-11</date>
        <extent unit="page">
            <start>163</start>
            <end>173</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Improving Natural Language Understanding by Reverse Mapping Bytepair Encoding
%A Tong, Chaodong
%A Peng, Huailiang
%A Dai, Qiong
%A Jiang, Lei
%A Huang, Jianghua
%Y Bansal, Mohit
%Y Villavicencio, Aline
%S Proceedings of the 23rd Conference on Computational Natural Language Learning (CoNLL)
%D 2019
%8 November
%I Association for Computational Linguistics
%C Hong Kong, China
%F tong-etal-2019-improving
%X We propose a method called reverse mapping bytepair encoding, which maps named-entity information and other word-level linguistic features back to subwords during the encoding procedure of bytepair encoding (BPE). We employ this method to the Generative Pre-trained Transformer (OpenAI GPT) by adding a weighted linear layer after the embedding layer. We also propose a new model architecture named as the multi-channel separate transformer to employ a training process without parameter-sharing. Evaluation on Stories Cloze, RTE, SciTail and SST-2 datasets demonstrates the effectiveness of our approach.
%R 10.18653/v1/K19-1016
%U https://aclanthology.org/K19-1016
%U https://doi.org/10.18653/v1/K19-1016
%P 163-173

Markdown (Informal)

[Improving Natural Language Understanding by Reverse Mapping Bytepair Encoding](https://aclanthology.org/K19-1016) (Tong et al., CoNLL 2019)

• Improving Natural Language Understanding by Reverse Mapping Bytepair Encoding (Tong et al., CoNLL 2019)

ACL

• Chaodong Tong, Huailiang Peng, Qiong Dai, Lei Jiang, and Jianghua Huang. 2019. Improving Natural Language Understanding by Reverse Mapping Bytepair Encoding. In Proceedings of the 23rd Conference on Computational Natural Language Learning (CoNLL), pages 163–173, Hong Kong, China. Association for Computational Linguistics.