<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="zhang-etal-2022-complicate">
    <titleInfo>
        <title>Complicate Then Simplify: A Novel Way to Explore Pre-trained Models for Text Classification</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Xu</namePart>
        <namePart type="family">Zhang</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Zejie</namePart>
        <namePart type="family">Liu</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Yanzheng</namePart>
        <namePart type="family">Xiang</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Deyu</namePart>
        <namePart type="family">Zhou</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2022-10</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the 29th International Conference on Computational Linguistics</title>
        </titleInfo>
        <originInfo>
            <publisher>International Committee on Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Gyeongju, Republic of Korea</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>With the development of pre-trained models (PTMs), the performance of text classification has been continuously improved by directly employing the features generated by PTMs. However such way might not fully explore the knowledge in PTMs as it is constrained by the difficulty of the task. Compared to difficult task, the learning algorithms tend to saturate early on the simple task. Moreover, the native sentence representations derived from BERT are prone to be collapsed and directly employing such representation for text classification might fail to fully capture discriminative features. In order to address these issues, in this paper we propose a novel framework for text classification which implements a two-stage training strategy. In the pre-training stage, auxiliary labels are introduced to increase the task difficulties and to fully exploit the knowledge in the pre-trained model. In the fine-tuning stage, the textual representation learned in the pre-training stage is employed and the classifier is fine-tuned to obtain better classification performance. Experiments were conducted on six text classification corpora and the results showed that the proposed framework outperformed several state-of-the-art baselines.</abstract>
    <identifier type="citekey">zhang-etal-2022-complicate</identifier>
    <location>
        <url>https://aclanthology.org/2022.coling-1.97</url>
    </location>
    <part>
        <date>2022-10</date>
        <extent unit="page">
            <start>1136</start>
            <end>1145</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Complicate Then Simplify: A Novel Way to Explore Pre-trained Models for Text Classification
%A Zhang, Xu
%A Liu, Zejie
%A Xiang, Yanzheng
%A Zhou, Deyu
%S Proceedings of the 29th International Conference on Computational Linguistics
%D 2022
%8 October
%I International Committee on Computational Linguistics
%C Gyeongju, Republic of Korea
%F zhang-etal-2022-complicate
%X With the development of pre-trained models (PTMs), the performance of text classification has been continuously improved by directly employing the features generated by PTMs. However such way might not fully explore the knowledge in PTMs as it is constrained by the difficulty of the task. Compared to difficult task, the learning algorithms tend to saturate early on the simple task. Moreover, the native sentence representations derived from BERT are prone to be collapsed and directly employing such representation for text classification might fail to fully capture discriminative features. In order to address these issues, in this paper we propose a novel framework for text classification which implements a two-stage training strategy. In the pre-training stage, auxiliary labels are introduced to increase the task difficulties and to fully exploit the knowledge in the pre-trained model. In the fine-tuning stage, the textual representation learned in the pre-training stage is employed and the classifier is fine-tuned to obtain better classification performance. Experiments were conducted on six text classification corpora and the results showed that the proposed framework outperformed several state-of-the-art baselines.
%U https://aclanthology.org/2022.coling-1.97
%P 1136-1145

Markdown (Informal)

[Complicate Then Simplify: A Novel Way to Explore Pre-trained Models for Text Classification](https://aclanthology.org/2022.coling-1.97) (Zhang et al., COLING 2022)

• Complicate Then Simplify: A Novel Way to Explore Pre-trained Models for Text Classification (Zhang et al., COLING 2022)

ACL

• Xu Zhang, Zejie Liu, Yanzheng Xiang, and Deyu Zhou. 2022. Complicate Then Simplify: A Novel Way to Explore Pre-trained Models for Text Classification. In Proceedings of the 29th International Conference on Computational Linguistics, pages 1136–1145, Gyeongju, Republic of Korea. International Committee on Computational Linguistics.