<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="zanoli-etal-2023-annotators">
    <titleInfo>
        <title>Annotators-in-the-loop: Testing a Novel Annotation Procedure on Italian Case Law</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Emma</namePart>
        <namePart type="family">Zanoli</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Matilde</namePart>
        <namePart type="family">Barbini</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Davide</namePart>
        <namePart type="family">Riva</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Sergio</namePart>
        <namePart type="family">Picascia</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Emanuela</namePart>
        <namePart type="family">Furiosi</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Stefano</namePart>
        <namePart type="family">D’Ancona</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Cristiano</namePart>
        <namePart type="family">Chesi</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2023-07</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the 17th Linguistic Annotation Workshop (LAW-XVII)</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Jakob</namePart>
            <namePart type="family">Prange</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Annemarie</namePart>
            <namePart type="family">Friedrich</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Toronto, Canada</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>The availability of annotated legal corpora is crucial for a number of tasks, such as legal search, legal information retrieval, and predictive justice. Annotation is mostly assumed to be a straightforward task: as long as the annotation scheme is well defined and the guidelines are clear, annotators are expected to agree on the labels. This is not always the case, especially in legal annotation, which can be extremely difficult even for expert annotators. We propose a legal annotation procedure that takes into account annotator certainty and improves it through negotiation. We also collect annotator feedback and show that our approach contributes to a positive annotation environment. Our work invites reflection on often neglected ethical concerns regarding legal annotation.</abstract>
    <identifier type="citekey">zanoli-etal-2023-annotators</identifier>
    <identifier type="doi">10.18653/v1/2023.law-1.12</identifier>
    <location>
        <url>https://aclanthology.org/2023.law-1.12</url>
    </location>
    <part>
        <date>2023-07</date>
        <extent unit="page">
            <start>118</start>
            <end>128</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Annotators-in-the-loop: Testing a Novel Annotation Procedure on Italian Case Law
%A Zanoli, Emma
%A Barbini, Matilde
%A Riva, Davide
%A Picascia, Sergio
%A Furiosi, Emanuela
%A D’Ancona, Stefano
%A Chesi, Cristiano
%Y Prange, Jakob
%Y Friedrich, Annemarie
%S Proceedings of the 17th Linguistic Annotation Workshop (LAW-XVII)
%D 2023
%8 July
%I Association for Computational Linguistics
%C Toronto, Canada
%F zanoli-etal-2023-annotators
%X The availability of annotated legal corpora is crucial for a number of tasks, such as legal search, legal information retrieval, and predictive justice. Annotation is mostly assumed to be a straightforward task: as long as the annotation scheme is well defined and the guidelines are clear, annotators are expected to agree on the labels. This is not always the case, especially in legal annotation, which can be extremely difficult even for expert annotators. We propose a legal annotation procedure that takes into account annotator certainty and improves it through negotiation. We also collect annotator feedback and show that our approach contributes to a positive annotation environment. Our work invites reflection on often neglected ethical concerns regarding legal annotation.
%R 10.18653/v1/2023.law-1.12
%U https://aclanthology.org/2023.law-1.12
%U https://doi.org/10.18653/v1/2023.law-1.12
%P 118-128

Markdown (Informal)

[Annotators-in-the-loop: Testing a Novel Annotation Procedure on Italian Case Law](https://aclanthology.org/2023.law-1.12) (Zanoli et al., LAW 2023)

• Annotators-in-the-loop: Testing a Novel Annotation Procedure on Italian Case Law (Zanoli et al., LAW 2023)

ACL

• Emma Zanoli, Matilde Barbini, Davide Riva, Sergio Picascia, Emanuela Furiosi, Stefano D’Ancona, and Cristiano Chesi. 2023. Annotators-in-the-loop: Testing a Novel Annotation Procedure on Italian Case Law. In Proceedings of the 17th Linguistic Annotation Workshop (LAW-XVII), pages 118–128, Toronto, Canada. Association for Computational Linguistics.