<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="neo-etal-2024-interpreting">
    <titleInfo>
        <title>Interpreting Context Look-ups in Transformers: Investigating Attention-MLP Interactions</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Clement</namePart>
        <namePart type="family">Neo</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Shay</namePart>
        <namePart type="family">Cohen</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Fazl</namePart>
        <namePart type="family">Barez</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2024-11</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Yaser</namePart>
            <namePart type="family">Al-Onaizan</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Mohit</namePart>
            <namePart type="family">Bansal</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Yun-Nung</namePart>
            <namePart type="family">Chen</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Miami, Florida, USA</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>Understanding the inner workings of large language models (LLMs) is crucial for advancing their theoretical foundations and real-world applications. While the attention mechanism and multi-layer perceptrons (MLPs) have been studied independently, their interactions remain largely unexplored. This study investigates how attention heads and next-token neurons interact in LLMs to predict new words. We propose a methodology to identify next-token neurons, find prompts that highly activate them, and determine the upstream attention heads responsible. We then generate and evaluate explanations for the activity of these attention heads in an automated manner. Our findings reveal that some attention heads recognize specific contexts relevant to predicting a token and activate a downstream token-predicting neuron accordingly. This mechanism provides a deeper understanding of how attention heads work with MLP neurons to perform next-token prediction. Our approach offers a foundation for further research into the intricate workings of LLMs and their impact on text generation and understanding.</abstract>
    <identifier type="citekey">neo-etal-2024-interpreting</identifier>
    <location>
        <url>https://aclanthology.org/2024.emnlp-main.930</url>
    </location>
    <part>
        <date>2024-11</date>
        <extent unit="page">
            <start>16681</start>
            <end>16697</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Interpreting Context Look-ups in Transformers: Investigating Attention-MLP Interactions
%A Neo, Clement
%A Cohen, Shay
%A Barez, Fazl
%Y Al-Onaizan, Yaser
%Y Bansal, Mohit
%Y Chen, Yun-Nung
%S Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing
%D 2024
%8 November
%I Association for Computational Linguistics
%C Miami, Florida, USA
%F neo-etal-2024-interpreting
%X Understanding the inner workings of large language models (LLMs) is crucial for advancing their theoretical foundations and real-world applications. While the attention mechanism and multi-layer perceptrons (MLPs) have been studied independently, their interactions remain largely unexplored. This study investigates how attention heads and next-token neurons interact in LLMs to predict new words. We propose a methodology to identify next-token neurons, find prompts that highly activate them, and determine the upstream attention heads responsible. We then generate and evaluate explanations for the activity of these attention heads in an automated manner. Our findings reveal that some attention heads recognize specific contexts relevant to predicting a token and activate a downstream token-predicting neuron accordingly. This mechanism provides a deeper understanding of how attention heads work with MLP neurons to perform next-token prediction. Our approach offers a foundation for further research into the intricate workings of LLMs and their impact on text generation and understanding.
%U https://aclanthology.org/2024.emnlp-main.930
%P 16681-16697

Markdown (Informal)

[Interpreting Context Look-ups in Transformers: Investigating Attention-MLP Interactions](https://aclanthology.org/2024.emnlp-main.930) (Neo et al., EMNLP 2024)

• Interpreting Context Look-ups in Transformers: Investigating Attention-MLP Interactions (Neo et al., EMNLP 2024)

ACL

• Clement Neo, Shay Cohen, and Fazl Barez. 2024. Interpreting Context Look-ups in Transformers: Investigating Attention-MLP Interactions. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 16681–16697, Miami, Florida, USA. Association for Computational Linguistics.