<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="guillot-etal-2023-sparser">
    <titleInfo>
        <title>Sparser is better: one step closer to word embedding interpretability</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Simon</namePart>
        <namePart type="family">Guillot</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Thibault</namePart>
        <namePart type="family">Prouteau</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Nicolas</namePart>
        <namePart type="family">Dugue</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2023-06</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the 15th International Conference on Computational Semantics</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Maxime</namePart>
            <namePart type="family">Amblard</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Ellen</namePart>
            <namePart type="family">Breitholtz</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Nancy, France</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>Sparse word embeddings models (SPINE, SINr) are designed to embed words in interpretable dimensions. An interpretable dimension is such that a human can interpret the semantic (or syntactic) relations between words active for a dimension. These models are useful for critical downstream tasks in natural language processing (e.g. medical or legal NLP), and digital humanities applications. This work extends interpretability at the vector level with a more manageable number of activated dimensions following recommendations from psycholinguistics. Subsequently, one of the key criteria to an interpretable model is sparsity: in order to be interpretable, not every word should be represented by all the features of the model, especially if humans have to interpret these features and their relations. This raises one question: to which extent is sparsity sustainable with regard to performance? We thus introduce a sparsification procedure to evaluate its impact on two interpretable methods (SPINE and SINr) to tend towards sustainable vector interpretability. We also introduce stability as a new criterion to interpretability. Our stability evaluations show little albeit non-zero variation for SPINE and SINr embeddings. We then show that increasing sparsity does not necessarily interfere with performance. These results are encouraging and pave the way towards intrinsically interpretable word vectors.</abstract>
    <identifier type="citekey">guillot-etal-2023-sparser</identifier>
    <location>
        <url>https://aclanthology.org/2023.iwcs-1.13</url>
    </location>
    <part>
        <date>2023-06</date>
        <extent unit="page">
            <start>106</start>
            <end>115</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Sparser is better: one step closer to word embedding interpretability
%A Guillot, Simon
%A Prouteau, Thibault
%A Dugue, Nicolas
%Y Amblard, Maxime
%Y Breitholtz, Ellen
%S Proceedings of the 15th International Conference on Computational Semantics
%D 2023
%8 June
%I Association for Computational Linguistics
%C Nancy, France
%F guillot-etal-2023-sparser
%X Sparse word embeddings models (SPINE, SINr) are designed to embed words in interpretable dimensions. An interpretable dimension is such that a human can interpret the semantic (or syntactic) relations between words active for a dimension. These models are useful for critical downstream tasks in natural language processing (e.g. medical or legal NLP), and digital humanities applications. This work extends interpretability at the vector level with a more manageable number of activated dimensions following recommendations from psycholinguistics. Subsequently, one of the key criteria to an interpretable model is sparsity: in order to be interpretable, not every word should be represented by all the features of the model, especially if humans have to interpret these features and their relations. This raises one question: to which extent is sparsity sustainable with regard to performance? We thus introduce a sparsification procedure to evaluate its impact on two interpretable methods (SPINE and SINr) to tend towards sustainable vector interpretability. We also introduce stability as a new criterion to interpretability. Our stability evaluations show little albeit non-zero variation for SPINE and SINr embeddings. We then show that increasing sparsity does not necessarily interfere with performance. These results are encouraging and pave the way towards intrinsically interpretable word vectors.
%U https://aclanthology.org/2023.iwcs-1.13
%P 106-115

Markdown (Informal)

[Sparser is better: one step closer to word embedding interpretability](https://aclanthology.org/2023.iwcs-1.13) (Guillot et al., IWCS 2023)

• Sparser is better: one step closer to word embedding interpretability (Guillot et al., IWCS 2023)

ACL

• Simon Guillot, Thibault Prouteau, and Nicolas Dugue. 2023. Sparser is better: one step closer to word embedding interpretability. In Proceedings of the 15th International Conference on Computational Semantics, pages 106–115, Nancy, France. Association for Computational Linguistics.