<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="mahmud-marculescu-2024-opensep">
    <titleInfo>
        <title>OpenSep: Leveraging Large Language Models with Textual Inversion for Open World Audio Separation</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Tanvir</namePart>
        <namePart type="family">Mahmud</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Diana</namePart>
        <namePart type="family">Marculescu</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2024-11</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Yaser</namePart>
            <namePart type="family">Al-Onaizan</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Mohit</namePart>
            <namePart type="family">Bansal</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Yun-Nung</namePart>
            <namePart type="family">Chen</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Miami, Florida, USA</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>Audio separation in real-world scenarios, where mixtures contain a variable number of sources, presents significant challenges due to limitations of existing models, such as over-separation, under-separation, and dependence on predefined training sources. We propose OpenSep, a novel framework that leverages large language models (LLMs) for automated audio separation, eliminating the need for manual intervention and overcoming source limitations. OpenSep uses textual inversion to generate captions from audio mixtures with off-the-shelf audio captioning models, effectively parsing the sound sources present. It then employs few-shot LLM prompting to extract detailed audio properties of each parsed source, facilitating separation in unseen mixtures. Additionally, we introduce a multi-level extension of the mix-and-separate training framework to enhance modality alignment by separating single source sounds and mixtures simultaneously. Extensive experiments demonstrate OpenSep’s superiority in precisely separating new, unseen, and variable sources in challenging mixtures, outperforming SOTA baseline methods. Code is released at https://github.com/tanvir-utexas/OpenSep.git.</abstract>
    <identifier type="citekey">mahmud-marculescu-2024-opensep</identifier>
    <location>
        <url>https://aclanthology.org/2024.emnlp-main.735</url>
    </location>
    <part>
        <date>2024-11</date>
        <extent unit="page">
            <start>13244</start>
            <end>13260</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T OpenSep: Leveraging Large Language Models with Textual Inversion for Open World Audio Separation
%A Mahmud, Tanvir
%A Marculescu, Diana
%Y Al-Onaizan, Yaser
%Y Bansal, Mohit
%Y Chen, Yun-Nung
%S Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing
%D 2024
%8 November
%I Association for Computational Linguistics
%C Miami, Florida, USA
%F mahmud-marculescu-2024-opensep
%X Audio separation in real-world scenarios, where mixtures contain a variable number of sources, presents significant challenges due to limitations of existing models, such as over-separation, under-separation, and dependence on predefined training sources. We propose OpenSep, a novel framework that leverages large language models (LLMs) for automated audio separation, eliminating the need for manual intervention and overcoming source limitations. OpenSep uses textual inversion to generate captions from audio mixtures with off-the-shelf audio captioning models, effectively parsing the sound sources present. It then employs few-shot LLM prompting to extract detailed audio properties of each parsed source, facilitating separation in unseen mixtures. Additionally, we introduce a multi-level extension of the mix-and-separate training framework to enhance modality alignment by separating single source sounds and mixtures simultaneously. Extensive experiments demonstrate OpenSep’s superiority in precisely separating new, unseen, and variable sources in challenging mixtures, outperforming SOTA baseline methods. Code is released at https://github.com/tanvir-utexas/OpenSep.git.
%U https://aclanthology.org/2024.emnlp-main.735
%P 13244-13260

Markdown (Informal)

[OpenSep: Leveraging Large Language Models with Textual Inversion for Open World Audio Separation](https://aclanthology.org/2024.emnlp-main.735) (Mahmud & Marculescu, EMNLP 2024)

• OpenSep: Leveraging Large Language Models with Textual Inversion for Open World Audio Separation (Mahmud & Marculescu, EMNLP 2024)

ACL

• Tanvir Mahmud and Diana Marculescu. 2024. OpenSep: Leveraging Large Language Models with Textual Inversion for Open World Audio Separation. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 13244–13260, Miami, Florida, USA. Association for Computational Linguistics.