<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="tint-2025-prood">
    <titleInfo>
        <title>PROOD: A Simple LLM Out-of-Distribution Guardrail Leveraging Response Semantics</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Joshua</namePart>
        <namePart type="family">Tint</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2025-11</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Findings of the Association for Computational Linguistics: EMNLP 2025</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Christos</namePart>
            <namePart type="family">Christodoulopoulos</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Tanmoy</namePart>
            <namePart type="family">Chakraborty</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Carolyn</namePart>
            <namePart type="family">Rose</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Violet</namePart>
            <namePart type="family">Peng</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Suzhou, China</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
        <identifier type="isbn">979-8-89176-335-7</identifier>
    </relatedItem>
    <abstract>Out-of-distribution (OOD) detection is a key safeguard for large language models, especially when they’re deployed in real-world applications. However, existing OOD methods often struggle with prompts that are deliberately obfuscated, context-dependent, or superficially benign—making it hard to distinguish between harmless queries and adversarial or dangerous ones. These methods typically assess prompts in isolation, missing important semantic cues from the model’s response. We introduce PROOD, prompt-response OOD detection, a framework that jointly analyzes LLM prompts and their corresponding outputs to improve semantic understanding. PROOD supports zero-shot multiclass detection using synthetic data generation and it offers a tunable probabilistic classification output. We validate PROOD on three challenging benchmarks—TrustLLM, OR-Bench, and AdvBench—where consistently outperforms prior OOD techniques, improving F1 scores by up to 6.3 points, from 0.871 to 0.934. Our results show that incorporating model responses enables more accurate, context-aware OOD detection in complex and adversarial prompt environments.</abstract>
    <identifier type="citekey">tint-2025-prood</identifier>
    <identifier type="doi">10.18653/v1/2025.findings-emnlp.1272</identifier>
    <location>
        <url>https://aclanthology.org/2025.findings-emnlp.1272/</url>
    </location>
    <part>
        <date>2025-11</date>
        <extent unit="page">
            <start>23428</start>
            <end>23438</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T PROOD: A Simple LLM Out-of-Distribution Guardrail Leveraging Response Semantics
%A Tint, Joshua
%Y Christodoulopoulos, Christos
%Y Chakraborty, Tanmoy
%Y Rose, Carolyn
%Y Peng, Violet
%S Findings of the Association for Computational Linguistics: EMNLP 2025
%D 2025
%8 November
%I Association for Computational Linguistics
%C Suzhou, China
%@ 979-8-89176-335-7
%F tint-2025-prood
%X Out-of-distribution (OOD) detection is a key safeguard for large language models, especially when they’re deployed in real-world applications. However, existing OOD methods often struggle with prompts that are deliberately obfuscated, context-dependent, or superficially benign—making it hard to distinguish between harmless queries and adversarial or dangerous ones. These methods typically assess prompts in isolation, missing important semantic cues from the model’s response. We introduce PROOD, prompt-response OOD detection, a framework that jointly analyzes LLM prompts and their corresponding outputs to improve semantic understanding. PROOD supports zero-shot multiclass detection using synthetic data generation and it offers a tunable probabilistic classification output. We validate PROOD on three challenging benchmarks—TrustLLM, OR-Bench, and AdvBench—where consistently outperforms prior OOD techniques, improving F1 scores by up to 6.3 points, from 0.871 to 0.934. Our results show that incorporating model responses enables more accurate, context-aware OOD detection in complex and adversarial prompt environments.
%R 10.18653/v1/2025.findings-emnlp.1272
%U https://aclanthology.org/2025.findings-emnlp.1272/
%U https://doi.org/10.18653/v1/2025.findings-emnlp.1272
%P 23428-23438

Markdown (Informal)

[PROOD: A Simple LLM Out-of-Distribution Guardrail Leveraging Response Semantics](https://aclanthology.org/2025.findings-emnlp.1272/) (Tint, Findings 2025)

• PROOD: A Simple LLM Out-of-Distribution Guardrail Leveraging Response Semantics (Tint, Findings 2025)

ACL

• Joshua Tint. 2025. PROOD: A Simple LLM Out-of-Distribution Guardrail Leveraging Response Semantics. In Findings of the Association for Computational Linguistics: EMNLP 2025, pages 23428–23438, Suzhou, China. Association for Computational Linguistics.