<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="elesedy-etal-2024-lora">
    <titleInfo>
        <title>LoRA-Guard: Parameter-Efficient Guardrail Adaptation for Content Moderation of Large Language Models</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Hayder</namePart>
        <namePart type="family">Elesedy</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Pedro</namePart>
        <namePart type="family">Esperanca</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Silviu</namePart>
        <namePart type="given">Vlad</namePart>
        <namePart type="family">Oprea</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Mete</namePart>
        <namePart type="family">Ozay</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2024-11</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Yaser</namePart>
            <namePart type="family">Al-Onaizan</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Mohit</namePart>
            <namePart type="family">Bansal</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Yun-Nung</namePart>
            <namePart type="family">Chen</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Miami, Florida, USA</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>Guardrails have emerged as an alternative to safety alignment for content moderation of large language models (LLMs). Existing model-based guardrails have not been designed for resource-constrained computational portable devices, such as mobile phones, more and more of which are running LLM-based applications locally. We introduce LoRA-Guard, a parameter-efficient guardrail adaptation method that relies on knowledge sharing between LLMs and guardrail models. LoRA-Guard extracts language features from the LLMs and adapts them for the content moderation task using low-rank adapters, while a dual-path design prevents any performance degradation on the generative task. We show that LoRA-Guard outperforms existing approaches with 100-1000x lower parameter overhead while maintaining accuracy, enabling on-device content moderation.</abstract>
    <identifier type="citekey">elesedy-etal-2024-lora</identifier>
    <location>
        <url>https://aclanthology.org/2024.emnlp-main.656</url>
    </location>
    <part>
        <date>2024-11</date>
        <extent unit="page">
            <start>11746</start>
            <end>11765</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T LoRA-Guard: Parameter-Efficient Guardrail Adaptation for Content Moderation of Large Language Models
%A Elesedy, Hayder
%A Esperanca, Pedro
%A Oprea, Silviu Vlad
%A Ozay, Mete
%Y Al-Onaizan, Yaser
%Y Bansal, Mohit
%Y Chen, Yun-Nung
%S Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing
%D 2024
%8 November
%I Association for Computational Linguistics
%C Miami, Florida, USA
%F elesedy-etal-2024-lora
%X Guardrails have emerged as an alternative to safety alignment for content moderation of large language models (LLMs). Existing model-based guardrails have not been designed for resource-constrained computational portable devices, such as mobile phones, more and more of which are running LLM-based applications locally. We introduce LoRA-Guard, a parameter-efficient guardrail adaptation method that relies on knowledge sharing between LLMs and guardrail models. LoRA-Guard extracts language features from the LLMs and adapts them for the content moderation task using low-rank adapters, while a dual-path design prevents any performance degradation on the generative task. We show that LoRA-Guard outperforms existing approaches with 100-1000x lower parameter overhead while maintaining accuracy, enabling on-device content moderation.
%U https://aclanthology.org/2024.emnlp-main.656
%P 11746-11765

Markdown (Informal)

[LoRA-Guard: Parameter-Efficient Guardrail Adaptation for Content Moderation of Large Language Models](https://aclanthology.org/2024.emnlp-main.656) (Elesedy et al., EMNLP 2024)

• LoRA-Guard: Parameter-Efficient Guardrail Adaptation for Content Moderation of Large Language Models (Elesedy et al., EMNLP 2024)

ACL

• Hayder Elesedy, Pedro Esperanca, Silviu Vlad Oprea, and Mete Ozay. 2024. LoRA-Guard: Parameter-Efficient Guardrail Adaptation for Content Moderation of Large Language Models. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 11746–11765, Miami, Florida, USA. Association for Computational Linguistics.