<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="schick-schutze-2021-shot">
    <titleInfo>
        <title>Few-Shot Text Generation with Natural Language Instructions</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Timo</namePart>
        <namePart type="family">Schick</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Hinrich</namePart>
        <namePart type="family">Schütze</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2021-11</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing</title>
        </titleInfo>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Online and Punta Cana, Dominican Republic</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>Providing pretrained language models with simple task descriptions in natural language enables them to solve some tasks in a fully unsupervised fashion. Moreover, when combined with regular learning from examples, this idea yields impressive few-shot results for a wide range of text classification tasks. It is also a promising direction to improve data efficiency in generative settings, but there are several challenges to using a combination of task descriptions and example-based learning for text generation. In particular, it is crucial to find task descriptions that are easy to understand for the pretrained model and to ensure that it actually makes good use of them; furthermore, effective measures against overfitting have to be implemented. In this paper, we show how these challenges can be tackled: We introduce GenPET, a method for text generation that is based on pattern-exploiting training, a recent approach for combining textual instructions with supervised learning that only works for classification tasks. On several summarization and headline generation datasets, GenPET gives consistent improvements over strong baselines in few-shot settings.</abstract>
    <identifier type="citekey">schick-schutze-2021-shot</identifier>
    <identifier type="doi">10.18653/v1/2021.emnlp-main.32</identifier>
    <location>
        <url>https://aclanthology.org/2021.emnlp-main.32</url>
    </location>
    <part>
        <date>2021-11</date>
        <extent unit="page">
            <start>390</start>
            <end>402</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Few-Shot Text Generation with Natural Language Instructions
%A Schick, Timo
%A Schütze, Hinrich
%S Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing
%D 2021
%8 November
%I Association for Computational Linguistics
%C Online and Punta Cana, Dominican Republic
%F schick-schutze-2021-shot
%X Providing pretrained language models with simple task descriptions in natural language enables them to solve some tasks in a fully unsupervised fashion. Moreover, when combined with regular learning from examples, this idea yields impressive few-shot results for a wide range of text classification tasks. It is also a promising direction to improve data efficiency in generative settings, but there are several challenges to using a combination of task descriptions and example-based learning for text generation. In particular, it is crucial to find task descriptions that are easy to understand for the pretrained model and to ensure that it actually makes good use of them; furthermore, effective measures against overfitting have to be implemented. In this paper, we show how these challenges can be tackled: We introduce GenPET, a method for text generation that is based on pattern-exploiting training, a recent approach for combining textual instructions with supervised learning that only works for classification tasks. On several summarization and headline generation datasets, GenPET gives consistent improvements over strong baselines in few-shot settings.
%R 10.18653/v1/2021.emnlp-main.32
%U https://aclanthology.org/2021.emnlp-main.32
%U https://doi.org/10.18653/v1/2021.emnlp-main.32
%P 390-402

Markdown (Informal)

[Few-Shot Text Generation with Natural Language Instructions](https://aclanthology.org/2021.emnlp-main.32) (Schick & Schütze, EMNLP 2021)

• Few-Shot Text Generation with Natural Language Instructions (Schick & Schütze, EMNLP 2021)

ACL

• Timo Schick and Hinrich Schütze. 2021. Few-Shot Text Generation with Natural Language Instructions. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 390–402, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics.