<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="divekar-durrett-2024-synthesizrr">
    <titleInfo>
        <title>SynthesizRR: Generating Diverse Datasets with Retrieval Augmentation</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Abhishek</namePart>
        <namePart type="family">Divekar</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Greg</namePart>
        <namePart type="family">Durrett</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2024-11</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Yaser</namePart>
            <namePart type="family">Al-Onaizan</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Mohit</namePart>
            <namePart type="family">Bansal</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Yun-Nung</namePart>
            <namePart type="family">Chen</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Miami, Florida, USA</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>It is often desirable to distill the capabilities of large language models (LLMs) into smaller student models due to compute and memory constraints. One way to do this for classification tasks is via dataset synthesis, which can be accomplished by generating examples of each label from the LLM. Prior approaches to synthesis use few-shot prompting, which relies on the LLM’s parametric knowledge to generate usable examples. However, this leads to issues of repetition, bias towards popular entities, and stylistic differences from human text. In this work, we propose Synthesize by Retrieval and Refinement (SynthesizRR), which uses retrieval augmentation to introduce variety into the dataset synthesis process: as retrieved passages vary, the LLM is seeded with different content to generate its examples. We empirically study the synthesis of six datasets, covering topic classification, sentiment analysis, tone detection, and humor, requiring complex synthesis strategies. We find SynthesizRR greatly improves lexical and semantic diversity, similarity to human-written text, and distillation performance, when compared to 32-shot prompting and four prior approaches.</abstract>
    <identifier type="citekey">divekar-durrett-2024-synthesizrr</identifier>
    <location>
        <url>https://aclanthology.org/2024.emnlp-main.1071</url>
    </location>
    <part>
        <date>2024-11</date>
        <extent unit="page">
            <start>19200</start>
            <end>19227</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T SynthesizRR: Generating Diverse Datasets with Retrieval Augmentation
%A Divekar, Abhishek
%A Durrett, Greg
%Y Al-Onaizan, Yaser
%Y Bansal, Mohit
%Y Chen, Yun-Nung
%S Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing
%D 2024
%8 November
%I Association for Computational Linguistics
%C Miami, Florida, USA
%F divekar-durrett-2024-synthesizrr
%X It is often desirable to distill the capabilities of large language models (LLMs) into smaller student models due to compute and memory constraints. One way to do this for classification tasks is via dataset synthesis, which can be accomplished by generating examples of each label from the LLM. Prior approaches to synthesis use few-shot prompting, which relies on the LLM’s parametric knowledge to generate usable examples. However, this leads to issues of repetition, bias towards popular entities, and stylistic differences from human text. In this work, we propose Synthesize by Retrieval and Refinement (SynthesizRR), which uses retrieval augmentation to introduce variety into the dataset synthesis process: as retrieved passages vary, the LLM is seeded with different content to generate its examples. We empirically study the synthesis of six datasets, covering topic classification, sentiment analysis, tone detection, and humor, requiring complex synthesis strategies. We find SynthesizRR greatly improves lexical and semantic diversity, similarity to human-written text, and distillation performance, when compared to 32-shot prompting and four prior approaches.
%U https://aclanthology.org/2024.emnlp-main.1071
%P 19200-19227

Markdown (Informal)

[SynthesizRR: Generating Diverse Datasets with Retrieval Augmentation](https://aclanthology.org/2024.emnlp-main.1071) (Divekar & Durrett, EMNLP 2024)

• SynthesizRR: Generating Diverse Datasets with Retrieval Augmentation (Divekar & Durrett, EMNLP 2024)

ACL

• Abhishek Divekar and Greg Durrett. 2024. SynthesizRR: Generating Diverse Datasets with Retrieval Augmentation. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 19200–19227, Miami, Florida, USA. Association for Computational Linguistics.