<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="zhong-etal-2026-revisiting">
    <titleInfo>
        <title>Revisiting Faithfulness Annotations for Long-form Summaries</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Yang</namePart>
        <namePart type="family">Zhong</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Yang</namePart>
        <namePart type="given">Janet</namePart>
        <namePart type="family">Liu</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Diane</namePart>
        <namePart type="family">Litman</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2026-07</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the 20th Linguistic Annotation Workshop (LAW XX)</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Yang</namePart>
            <namePart type="given">Janet</namePart>
            <namePart type="family">Liu</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Luke</namePart>
            <namePart type="family">Gessler</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">San Diego, California, USA</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
        <identifier type="isbn">979-8-89176-404-0</identifier>
    </relatedItem>
    <abstract>Benchmarks for long-form summaries (four or more sentences) generated by language models increasingly serve as gold-standard references for developing, evaluating, and comparing faithfulness-checking systems. As their influence grows, understanding the challenges of annotating faithfulness errors within long, discourse-rich summaries becomes critical. We revisit three benchmarks spanning diverse text types and contrasting annotation designs. Using a discourse-aware evaluation framework together with human auditing, we identify cases where benchmark labels may be unreliable. Manual verification shows that 3.4%-5.4% of sentence-level labels warrant revision due to discourse-level inconsistencies that standard annotation procedures overlook. We introduce a taxonomy of five recurring annotation error types, propose revised labels, and show that correcting these cases leads to meaningful shifts in system rankings. We conclude with recommendations for future annotation practices.</abstract>
    <identifier type="citekey">zhong-etal-2026-revisiting</identifier>
    <location>
        <url>https://aclanthology.org/2026.law-main.12/</url>
    </location>
    <part>
        <date>2026-07</date>
        <extent unit="page">
            <start>158</start>
            <end>172</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Revisiting Faithfulness Annotations for Long-form Summaries
%A Zhong, Yang
%A Liu, Yang Janet
%A Litman, Diane
%Y Liu, Yang Janet
%Y Gessler, Luke
%S Proceedings of the 20th Linguistic Annotation Workshop (LAW XX)
%D 2026
%8 July
%I Association for Computational Linguistics
%C San Diego, California, USA
%@ 979-8-89176-404-0
%F zhong-etal-2026-revisiting
%X Benchmarks for long-form summaries (four or more sentences) generated by language models increasingly serve as gold-standard references for developing, evaluating, and comparing faithfulness-checking systems. As their influence grows, understanding the challenges of annotating faithfulness errors within long, discourse-rich summaries becomes critical. We revisit three benchmarks spanning diverse text types and contrasting annotation designs. Using a discourse-aware evaluation framework together with human auditing, we identify cases where benchmark labels may be unreliable. Manual verification shows that 3.4%-5.4% of sentence-level labels warrant revision due to discourse-level inconsistencies that standard annotation procedures overlook. We introduce a taxonomy of five recurring annotation error types, propose revised labels, and show that correcting these cases leads to meaningful shifts in system rankings. We conclude with recommendations for future annotation practices.
%U https://aclanthology.org/2026.law-main.12/
%P 158-172

Markdown (Informal)

[Revisiting Faithfulness Annotations for Long-form Summaries](https://aclanthology.org/2026.law-main.12/) (Zhong et al., LAW 2026)

• Revisiting Faithfulness Annotations for Long-form Summaries (Zhong et al., LAW 2026)

ACL

• Yang Zhong, Yang Janet Liu, and Diane Litman. 2026. Revisiting Faithfulness Annotations for Long-form Summaries. In Proceedings of the 20th Linguistic Annotation Workshop (LAW XX), pages 158–172, San Diego, California, USA. Association for Computational Linguistics.