SubmissionNumber#=%=#13 FinalPaperTitle#=%=#From Sparse to Dense: GPT-4 Summarization with Chain of Density Prompting ShortPaperTitle#=%=# NumberOfPages#=%=#7 CopyrightSigned#=%=#Griffin Adams JobTitle#==# Organization#==# Abstract#==#Selecting the ``right'' amount of information to include in a summary is a difficult task. A good summary should be detailed and entity-centric without being overly dense and hard to follow. To better understand this tradeoff, we solicit increasingly dense GPT-4 summaries with what we refer to as a ``Chain of Density'' (CoD) prompt. Specifically, GPT-4 generates an initial entity-sparse summary before iteratively incorporating missing salient entities without increasing the length. Summaries generated by CoD are more abstractive, exhibit more fusion, and have less of a lead bias than GPT-4 summaries generated by a vanilla prompt. We conduct a human preference study on 100 CNN DailyMail articles and find that humans prefer GPT-4 summaries that are more dense than those generated by a vanilla prompt and almost as dense as human written summaries. Qualitative analysis supports the notion that there exists a tradeoff between informativeness and readability. 500 annotated CoD summaries, as well as an extra 5,000 unannotated summaries, are freely available on HuggingFace (https://huggingface.co/datasets/griffin/chain_of_density). Author{1}{Firstname}#=%=#Griffin Author{1}{Lastname}#=%=#Adams Author{1}{Username}#=%=#griffinadams Author{1}{Email}#=%=#griffin.adams@columbia.edu Author{1}{Affiliation}#=%=#Columbia University Author{2}{Firstname}#=%=#Alex Author{2}{Lastname}#=%=#Fabbri Author{2}{Username}#=%=#alexfabbri Author{2}{Email}#=%=#afabbri@salesforce.com Author{2}{Affiliation}#=%=#Salesforce AI Research Author{3}{Firstname}#=%=#Faisal Author{3}{Lastname}#=%=#Ladhak Author{3}{Username}#=%=#kvothe Author{3}{Email}#=%=#faisal@cs.columbia.edu Author{3}{Affiliation}#=%=#Columbia University Author{4}{Firstname}#=%=#Eric Author{4}{Lastname}#=%=#Lehman Author{4}{Username}#=%=#lehmer16 Author{4}{Email}#=%=#lehmer16@mit.edu Author{4}{Affiliation}#=%=#MIT Author{5}{Firstname}#=%=#Noémie Author{5}{Lastname}#=%=#Elhadad Author{5}{Username}#=%=#noemie Author{5}{Email}#=%=#noemie.elhadad@columbia.edu Author{5}{Affiliation}#=%=#Columbia University ========== èéáğö