<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="yang-shen-2023-boosting">
    <titleInfo>
        <title>Boosting Summarization with Normalizing Flows and Aggressive Training</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Yu</namePart>
        <namePart type="family">Yang</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Xiaotong</namePart>
        <namePart type="family">Shen</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2023-12</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Houda</namePart>
            <namePart type="family">Bouamor</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Juan</namePart>
            <namePart type="family">Pino</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Kalika</namePart>
            <namePart type="family">Bali</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Singapore</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>This paper presents FlowSUM, a normalizing flows-based variational encoder-decoder framework for Transformer-based summarization. Our approach tackles two primary challenges in variational summarization: insufficient semantic information in latent representations and posterior collapse during training. To address these challenges, we employ normalizing flows to enable flexible latent posterior modeling, and we propose a controlled alternate aggressive training (CAAT) strategy with an improved gate mechanism. Experimental results show that FlowSUM significantly enhances the quality of generated summaries and unleashes the potential for knowledge distillation with minimal impact on inference time. Furthermore, we investigate the issue of posterior collapse in normalizing flows and analyze how the summary quality is affected by the training strategy, gate initialization, and the type and number of normalizing flows used, offering valuable insights for future research.</abstract>
    <identifier type="citekey">yang-shen-2023-boosting</identifier>
    <identifier type="doi">10.18653/v1/2023.emnlp-main.165</identifier>
    <location>
        <url>https://aclanthology.org/2023.emnlp-main.165</url>
    </location>
    <part>
        <date>2023-12</date>
        <extent unit="page">
            <start>2727</start>
            <end>2751</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Boosting Summarization with Normalizing Flows and Aggressive Training
%A Yang, Yu
%A Shen, Xiaotong
%Y Bouamor, Houda
%Y Pino, Juan
%Y Bali, Kalika
%S Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing
%D 2023
%8 December
%I Association for Computational Linguistics
%C Singapore
%F yang-shen-2023-boosting
%X This paper presents FlowSUM, a normalizing flows-based variational encoder-decoder framework for Transformer-based summarization. Our approach tackles two primary challenges in variational summarization: insufficient semantic information in latent representations and posterior collapse during training. To address these challenges, we employ normalizing flows to enable flexible latent posterior modeling, and we propose a controlled alternate aggressive training (CAAT) strategy with an improved gate mechanism. Experimental results show that FlowSUM significantly enhances the quality of generated summaries and unleashes the potential for knowledge distillation with minimal impact on inference time. Furthermore, we investigate the issue of posterior collapse in normalizing flows and analyze how the summary quality is affected by the training strategy, gate initialization, and the type and number of normalizing flows used, offering valuable insights for future research.
%R 10.18653/v1/2023.emnlp-main.165
%U https://aclanthology.org/2023.emnlp-main.165
%U https://doi.org/10.18653/v1/2023.emnlp-main.165
%P 2727-2751

Markdown (Informal)

[Boosting Summarization with Normalizing Flows and Aggressive Training](https://aclanthology.org/2023.emnlp-main.165) (Yang & Shen, EMNLP 2023)

• Boosting Summarization with Normalizing Flows and Aggressive Training (Yang & Shen, EMNLP 2023)

ACL

• Yu Yang and Xiaotong Shen. 2023. Boosting Summarization with Normalizing Flows and Aggressive Training. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 2727–2751, Singapore. Association for Computational Linguistics.