<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="hawasly-etal-2024-scaling">
    <titleInfo>
        <title>Scaling up Discovery of Latent Concepts in Deep NLP Models</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Majd</namePart>
        <namePart type="family">Hawasly</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Fahim</namePart>
        <namePart type="family">Dalvi</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Nadir</namePart>
        <namePart type="family">Durrani</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2024-03</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers)</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Yvette</namePart>
            <namePart type="family">Graham</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Matthew</namePart>
            <namePart type="family">Purver</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">St. Julian’s, Malta</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>Despite the revolution caused by deep NLP models, they remain black boxes, necessitating research to understand their decision-making processes. A recent work by Dalvi et al. (2022) carried out representation analysis through the lens of clustering latent spaces within pre-trained models (PLMs), but that approach is limited to small scale due to the high cost of running Agglomerative hierarchical clustering. This paper studies clustering algorithms in order to scale the discovery of encoded concepts in PLM representations to larger datasets and models. We propose metrics for assessing the quality of discovered latent concepts and use them to compare the studied clustering algorithms. We found that K-Means-based concept discovery significantly enhances efficiency while maintaining the quality of the obtained concepts. Furthermore, we demonstrate the practicality of this newfound efficiency by scaling latent concept discovery to LLMs and phrasal concepts.</abstract>
    <identifier type="citekey">hawasly-etal-2024-scaling</identifier>
    <location>
        <url>https://aclanthology.org/2024.eacl-long.48</url>
    </location>
    <part>
        <date>2024-03</date>
        <extent unit="page">
            <start>793</start>
            <end>806</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Scaling up Discovery of Latent Concepts in Deep NLP Models
%A Hawasly, Majd
%A Dalvi, Fahim
%A Durrani, Nadir
%Y Graham, Yvette
%Y Purver, Matthew
%S Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers)
%D 2024
%8 March
%I Association for Computational Linguistics
%C St. Julian’s, Malta
%F hawasly-etal-2024-scaling
%X Despite the revolution caused by deep NLP models, they remain black boxes, necessitating research to understand their decision-making processes. A recent work by Dalvi et al. (2022) carried out representation analysis through the lens of clustering latent spaces within pre-trained models (PLMs), but that approach is limited to small scale due to the high cost of running Agglomerative hierarchical clustering. This paper studies clustering algorithms in order to scale the discovery of encoded concepts in PLM representations to larger datasets and models. We propose metrics for assessing the quality of discovered latent concepts and use them to compare the studied clustering algorithms. We found that K-Means-based concept discovery significantly enhances efficiency while maintaining the quality of the obtained concepts. Furthermore, we demonstrate the practicality of this newfound efficiency by scaling latent concept discovery to LLMs and phrasal concepts.
%U https://aclanthology.org/2024.eacl-long.48
%P 793-806

Markdown (Informal)

[Scaling up Discovery of Latent Concepts in Deep NLP Models](https://aclanthology.org/2024.eacl-long.48) (Hawasly et al., EACL 2024)

• Scaling up Discovery of Latent Concepts in Deep NLP Models (Hawasly et al., EACL 2024)

ACL

• Majd Hawasly, Fahim Dalvi, and Nadir Durrani. 2024. Scaling up Discovery of Latent Concepts in Deep NLP Models. In Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers), pages 793–806, St. Julian’s, Malta. Association for Computational Linguistics.