<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="lent-sogaard-2021-common">
    <titleInfo>
        <title>Common Sense Bias in Semantic Role Labeling</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Heather</namePart>
        <namePart type="family">Lent</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Anders</namePart>
        <namePart type="family">Søgaard</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2021-11</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the Seventh Workshop on Noisy User-generated Text (W-NUT 2021)</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Wei</namePart>
            <namePart type="family">Xu</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Alan</namePart>
            <namePart type="family">Ritter</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Tim</namePart>
            <namePart type="family">Baldwin</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Afshin</namePart>
            <namePart type="family">Rahimi</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Online</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>Large-scale language models such as ELMo and BERT have pushed the horizon of what is possible in semantic role labeling (SRL), solving the out-of-vocabulary problem and enabling end-to-end systems, but they have also introduced significant biases. We evaluate three SRL parsers on very simple transitive sentences with verbs usually associated with animate subjects and objects, such as, “Mary babysat Tom”: a state-of-the-art parser based on BERT, an older parser based on GloVe, and an even older parser from before the days of word embeddings. When arguments are word forms predominantly used as person names, aligning with common sense expectations of animacy, the BERT-based parser is unsurprisingly superior; yet, with abstract or random nouns, the opposite picture emerges. We refer to this as “common sense bias” and present a challenge dataset for evaluating the extent to which parsers are sensitive to such a bias. Our code and challenge dataset are available here: github.com/coastalcph/comte</abstract>
    <identifier type="citekey">lent-sogaard-2021-common</identifier>
    <identifier type="doi">10.18653/v1/2021.wnut-1.14</identifier>
    <location>
        <url>https://aclanthology.org/2021.wnut-1.14</url>
    </location>
    <part>
        <date>2021-11</date>
        <extent unit="page">
            <start>114</start>
            <end>119</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Common Sense Bias in Semantic Role Labeling
%A Lent, Heather
%A Søgaard, Anders
%Y Xu, Wei
%Y Ritter, Alan
%Y Baldwin, Tim
%Y Rahimi, Afshin
%S Proceedings of the Seventh Workshop on Noisy User-generated Text (W-NUT 2021)
%D 2021
%8 November
%I Association for Computational Linguistics
%C Online
%F lent-sogaard-2021-common
%X Large-scale language models such as ELMo and BERT have pushed the horizon of what is possible in semantic role labeling (SRL), solving the out-of-vocabulary problem and enabling end-to-end systems, but they have also introduced significant biases. We evaluate three SRL parsers on very simple transitive sentences with verbs usually associated with animate subjects and objects, such as, “Mary babysat Tom”: a state-of-the-art parser based on BERT, an older parser based on GloVe, and an even older parser from before the days of word embeddings. When arguments are word forms predominantly used as person names, aligning with common sense expectations of animacy, the BERT-based parser is unsurprisingly superior; yet, with abstract or random nouns, the opposite picture emerges. We refer to this as “common sense bias” and present a challenge dataset for evaluating the extent to which parsers are sensitive to such a bias. Our code and challenge dataset are available here: github.com/coastalcph/comte
%R 10.18653/v1/2021.wnut-1.14
%U https://aclanthology.org/2021.wnut-1.14
%U https://doi.org/10.18653/v1/2021.wnut-1.14
%P 114-119

Markdown (Informal)

[Common Sense Bias in Semantic Role Labeling](https://aclanthology.org/2021.wnut-1.14) (Lent & Søgaard, WNUT 2021)

• Common Sense Bias in Semantic Role Labeling (Lent & Søgaard, WNUT 2021)

ACL

• Heather Lent and Anders Søgaard. 2021. Common Sense Bias in Semantic Role Labeling. In Proceedings of the Seventh Workshop on Noisy User-generated Text (W-NUT 2021), pages 114–119, Online. Association for Computational Linguistics.