<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="shi-etal-2024-structured">
    <titleInfo>
        <title>Structured Tree Alignment for Evaluation of (Speech) Constituency Parsing</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Freda</namePart>
        <namePart type="family">Shi</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Kevin</namePart>
        <namePart type="family">Gimpel</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Karen</namePart>
        <namePart type="family">Livescu</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2024-08</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Lun-Wei</namePart>
            <namePart type="family">Ku</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Andre</namePart>
            <namePart type="family">Martins</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Vivek</namePart>
            <namePart type="family">Srikumar</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Bangkok, Thailand</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>We present the structured average intersection-over-union ratio (STRUCT-IOU), an evaluation metric that compares a constituency parse tree over automatically recognized spoken word boundaries with the ground-truth parse tree over written words. To compute the metric, we (1) project the ground-truth parse tree to the speech domain by forced alignment, (2) align the projected ground-truth constituents with the predicted ones under certain structured constraints, and (3) calculate the average IOU score across all aligned constituent pairs. STRUCT-IOU takes word boundaries into account and overcomes the challenge that the predicted words and ground truth may not have perfect one-to-one correspondence. Extending to the evaluation of text constituency parsing, we demonstrate that STRUCT-IOU shows higher tolerance to syntactically plausible parses than PARSEVAL (Black et al., 1991).</abstract>
    <identifier type="citekey">shi-etal-2024-structured</identifier>
    <identifier type="doi">10.18653/v1/2024.acl-long.666</identifier>
    <location>
        <url>https://aclanthology.org/2024.luhme-long.666/</url>
    </location>
    <part>
        <date>2024-08</date>
        <extent unit="page">
            <start>12320</start>
            <end>12332</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Structured Tree Alignment for Evaluation of (Speech) Constituency Parsing
%A Shi, Freda
%A Gimpel, Kevin
%A Livescu, Karen
%Y Ku, Lun-Wei
%Y Martins, Andre
%Y Srikumar, Vivek
%S Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
%D 2024
%8 August
%I Association for Computational Linguistics
%C Bangkok, Thailand
%F shi-etal-2024-structured
%X We present the structured average intersection-over-union ratio (STRUCT-IOU), an evaluation metric that compares a constituency parse tree over automatically recognized spoken word boundaries with the ground-truth parse tree over written words. To compute the metric, we (1) project the ground-truth parse tree to the speech domain by forced alignment, (2) align the projected ground-truth constituents with the predicted ones under certain structured constraints, and (3) calculate the average IOU score across all aligned constituent pairs. STRUCT-IOU takes word boundaries into account and overcomes the challenge that the predicted words and ground truth may not have perfect one-to-one correspondence. Extending to the evaluation of text constituency parsing, we demonstrate that STRUCT-IOU shows higher tolerance to syntactically plausible parses than PARSEVAL (Black et al., 1991).
%R 10.18653/v1/2024.acl-long.666
%U https://aclanthology.org/2024.luhme-long.666/
%U https://doi.org/10.18653/v1/2024.acl-long.666
%P 12320-12332

Markdown (Informal)

[Structured Tree Alignment for Evaluation of (Speech) Constituency Parsing](https://aclanthology.org/2024.luhme-long.666/) (Shi et al., ACL 2024)

• Structured Tree Alignment for Evaluation of (Speech) Constituency Parsing (Shi et al., ACL 2024)

ACL

• Freda Shi, Kevin Gimpel, and Karen Livescu. 2024. Structured Tree Alignment for Evaluation of (Speech) Constituency Parsing. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 12320–12332, Bangkok, Thailand. Association for Computational Linguistics.