<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="trimigno-etal-2026-anchorlens">
    <titleInfo>
        <title>AnchorLens: A Shared Coordinate System for Interpreting Intermediate Representations in Large Language Models</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Giuseppe</namePart>
        <namePart type="family">Trimigno</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Gianfranco</namePart>
        <namePart type="family">Lombardo</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Stefano</namePart>
        <namePart type="family">Cagnoni</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2026-10</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Hung-Yi</namePart>
            <namePart type="family">Lee</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Sunipa</namePart>
            <namePart type="family">Dev</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Jessy</namePart>
            <namePart type="family">Li</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Desmond</namePart>
            <namePart type="family">Elliott</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Budapest, Hungary</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
        <identifier type="isbn">979-8-89176-459-0</identifier>
    </relatedItem>
    <abstract>Lens methods aim to decode intermediate representations of large language models into vocabulary predictions by training independent translators for each layer. While this approach provides insights into how the final response emerges across layers, it overlooks the dominant structure shared among them. We identify that per-layer alignment problems share a dominant common structure: the first singular component of per-layer translators explains 31–64% of total spectral variance, 10–20\times above the unstructured baseline. Motivated by this evidence, we propose AnchorLens, which exploits a global-local decomposition: a shared full-rank matrix G maps layers into a common coordinate system, while per-layer low-rank adapters correct layer-specific residuals, trained in two stages. AnchorLens achieves over 90% parameter reduction relative to full-rank methods while matching TunedLens and outperforming LoRALens on next-token prediction and linear shortcut evaluations across eight benchmark tasks. These results hold across 16 models up to 120B parameters from five families: GPT-OSS, Qwen3, Qwen2.5, OLMo2, and Gemma2. Furthermore, the shared coordinate system induced by G enables geometric analyses inaccessible to independent per-layer methods, including quantification of inter-layer drift reduction and empirical validation of non-degeneracy: post-training adapter orthogonality confirms G absorbs the shared structure. Our code is publicly available on https://github.com/GiuseT/InterpretabilityLenses.git.</abstract>
    <identifier type="citekey">trimigno-etal-2026-anchorlens</identifier>
    <location>
        <url>https://aclanthology.org/2026.emnlp-main.2385/</url>
    </location>
    <part>
        <date>2026-10</date>
        <extent unit="page">
            <start>54183</start>
            <end>54208</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T AnchorLens: A Shared Coordinate System for Interpreting Intermediate Representations in Large Language Models
%A Trimigno, Giuseppe
%A Lombardo, Gianfranco
%A Cagnoni, Stefano
%Y Lee, Hung-Yi
%Y Dev, Sunipa
%Y Li, Jessy
%Y Elliott, Desmond
%S Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
%D 2026
%8 October
%I Association for Computational Linguistics
%C Budapest, Hungary
%@ 979-8-89176-459-0
%F trimigno-etal-2026-anchorlens
%X Lens methods aim to decode intermediate representations of large language models into vocabulary predictions by training independent translators for each layer. While this approach provides insights into how the final response emerges across layers, it overlooks the dominant structure shared among them. We identify that per-layer alignment problems share a dominant common structure: the first singular component of per-layer translators explains 31–64% of total spectral variance, 10–20\times above the unstructured baseline. Motivated by this evidence, we propose AnchorLens, which exploits a global-local decomposition: a shared full-rank matrix G maps layers into a common coordinate system, while per-layer low-rank adapters correct layer-specific residuals, trained in two stages. AnchorLens achieves over 90% parameter reduction relative to full-rank methods while matching TunedLens and outperforming LoRALens on next-token prediction and linear shortcut evaluations across eight benchmark tasks. These results hold across 16 models up to 120B parameters from five families: GPT-OSS, Qwen3, Qwen2.5, OLMo2, and Gemma2. Furthermore, the shared coordinate system induced by G enables geometric analyses inaccessible to independent per-layer methods, including quantification of inter-layer drift reduction and empirical validation of non-degeneracy: post-training adapter orthogonality confirms G absorbs the shared structure. Our code is publicly available on https://github.com/GiuseT/InterpretabilityLenses.git.
%U https://aclanthology.org/2026.emnlp-main.2385/
%P 54183-54208

Markdown (Informal)

[AnchorLens: A Shared Coordinate System for Interpreting Intermediate Representations in Large Language Models](https://aclanthology.org/2026.emnlp-main.2385/) (Trimigno et al., EMNLP 2026)

• AnchorLens: A Shared Coordinate System for Interpreting Intermediate Representations in Large Language Models (Trimigno et al., EMNLP 2026)

ACL

• Giuseppe Trimigno, Gianfranco Lombardo, and Stefano Cagnoni. 2026. AnchorLens: A Shared Coordinate System for Interpreting Intermediate Representations in Large Language Models. In Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing, pages 54183–54208, Budapest, Hungary. Association for Computational Linguistics.