<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="ramnauth-scassellati-2026-attentional">
    <titleInfo>
        <title>The Attentional White Bear: Persistent But Attenuated Concept Recoverability in Transformer Language Models</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Rebecca</namePart>
        <namePart type="family">Ramnauth</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Brian</namePart>
        <namePart type="family">Scassellati</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2026-10</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Hung-Yi</namePart>
            <namePart type="family">Lee</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Sunipa</namePart>
            <namePart type="family">Dev</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Jessy</namePart>
            <namePart type="family">Li</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Desmond</namePart>
            <namePart type="family">Elliott</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Budapest, Hungary</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
        <identifier type="isbn">979-8-89176-459-0</identifier>
    </relatedItem>
    <abstract>Instruction-based suppression is widely used to prevent language models from generating prohibited content, yet it remains unclear whether suppression reduces internal representation or merely suppresses expression. We investigate this question through representational probing, attention analysis, and behavioral semantic leakage experiments across multiple transformer models. We find that prohibited concepts remain highly recoverable from hidden representations under suppression, influence localized changes in attention routing, and measurably shape downstream generations despite successful lexical avoidance. These effects persist across pooling strategies, indirect semantic controls, and multiple model families. Our results expose a fundamental gap between behavioral and representational alignment.</abstract>
    <identifier type="citekey">ramnauth-scassellati-2026-attentional</identifier>
    <location>
        <url>https://aclanthology.org/2026.emnlp-main.1609/</url>
    </location>
    <part>
        <date>2026-10</date>
        <extent unit="page">
            <start>36600</start>
            <end>36614</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T The Attentional White Bear: Persistent But Attenuated Concept Recoverability in Transformer Language Models
%A Ramnauth, Rebecca
%A Scassellati, Brian
%Y Lee, Hung-Yi
%Y Dev, Sunipa
%Y Li, Jessy
%Y Elliott, Desmond
%S Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
%D 2026
%8 October
%I Association for Computational Linguistics
%C Budapest, Hungary
%@ 979-8-89176-459-0
%F ramnauth-scassellati-2026-attentional
%X Instruction-based suppression is widely used to prevent language models from generating prohibited content, yet it remains unclear whether suppression reduces internal representation or merely suppresses expression. We investigate this question through representational probing, attention analysis, and behavioral semantic leakage experiments across multiple transformer models. We find that prohibited concepts remain highly recoverable from hidden representations under suppression, influence localized changes in attention routing, and measurably shape downstream generations despite successful lexical avoidance. These effects persist across pooling strategies, indirect semantic controls, and multiple model families. Our results expose a fundamental gap between behavioral and representational alignment.
%U https://aclanthology.org/2026.emnlp-main.1609/
%P 36600-36614

Markdown (Informal)

[The Attentional White Bear: Persistent But Attenuated Concept Recoverability in Transformer Language Models](https://aclanthology.org/2026.emnlp-main.1609/) (Ramnauth & Scassellati, EMNLP 2026)

• The Attentional White Bear: Persistent But Attenuated Concept Recoverability in Transformer Language Models (Ramnauth & Scassellati, EMNLP 2026)

ACL

• Rebecca Ramnauth and Brian Scassellati. 2026. The Attentional White Bear: Persistent But Attenuated Concept Recoverability in Transformer Language Models. In Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing, pages 36600–36614, Budapest, Hungary. Association for Computational Linguistics.