<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="morger-etal-2022-cross">
    <titleInfo>
        <title>A Cross-lingual Comparison of Human and Model Relative Word Importance</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Felix</namePart>
        <namePart type="family">Morger</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Stephanie</namePart>
        <namePart type="family">Brandl</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Lisa</namePart>
        <namePart type="family">Beinborn</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Nora</namePart>
        <namePart type="family">Hollenstein</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2022-09</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the 2022 CLASP Conference on (Dis)embodiment</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Simon</namePart>
            <namePart type="family">Dobnik</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Julian</namePart>
            <namePart type="family">Grove</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Asad</namePart>
            <namePart type="family">Sayeed</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Gothenburg, Sweden</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>Relative word importance is a key metric for natural language processing. In this work, we compare human and model relative word importance to investigate if pretrained neural language models focus on the same words as humans cross-lingually. We perform an extensive study using several importance metrics (gradient-based saliency and attention-based) in monolingual and multilingual models, including eye-tracking corpora from four languages (German, Dutch, English, and Russian). We find that gradient-based saliency, first-layer attention, and attention flow correlate strongly with human eye-tracking data across all four languages. We further analyze the role of word length and word frequency in determining relative importance and find that it strongly correlates with length and frequency, however, the mechanisms behind these non-linear relations remain elusive. We obtain a cross-lingual approximation of the similarity between human and computational language processing and insights into the usability of several importance metrics.</abstract>
    <identifier type="citekey">morger-etal-2022-cross</identifier>
    <location>
        <url>https://aclanthology.org/2022.clasp-1.2</url>
    </location>
    <part>
        <date>2022-09</date>
        <extent unit="page">
            <start>11</start>
            <end>23</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T A Cross-lingual Comparison of Human and Model Relative Word Importance
%A Morger, Felix
%A Brandl, Stephanie
%A Beinborn, Lisa
%A Hollenstein, Nora
%Y Dobnik, Simon
%Y Grove, Julian
%Y Sayeed, Asad
%S Proceedings of the 2022 CLASP Conference on (Dis)embodiment
%D 2022
%8 September
%I Association for Computational Linguistics
%C Gothenburg, Sweden
%F morger-etal-2022-cross
%X Relative word importance is a key metric for natural language processing. In this work, we compare human and model relative word importance to investigate if pretrained neural language models focus on the same words as humans cross-lingually. We perform an extensive study using several importance metrics (gradient-based saliency and attention-based) in monolingual and multilingual models, including eye-tracking corpora from four languages (German, Dutch, English, and Russian). We find that gradient-based saliency, first-layer attention, and attention flow correlate strongly with human eye-tracking data across all four languages. We further analyze the role of word length and word frequency in determining relative importance and find that it strongly correlates with length and frequency, however, the mechanisms behind these non-linear relations remain elusive. We obtain a cross-lingual approximation of the similarity between human and computational language processing and insights into the usability of several importance metrics.
%U https://aclanthology.org/2022.clasp-1.2
%P 11-23

Markdown (Informal)

[A Cross-lingual Comparison of Human and Model Relative Word Importance](https://aclanthology.org/2022.clasp-1.2) (Morger et al., CLASP 2022)

• A Cross-lingual Comparison of Human and Model Relative Word Importance (Morger et al., CLASP 2022)

ACL

• Felix Morger, Stephanie Brandl, Lisa Beinborn, and Nora Hollenstein. 2022. A Cross-lingual Comparison of Human and Model Relative Word Importance. In Proceedings of the 2022 CLASP Conference on (Dis)embodiment, pages 11–23, Gothenburg, Sweden. Association for Computational Linguistics.