<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="yang-etal-2026-string">
    <titleInfo>
        <title>When String Matching Stands In for a Knowledge Base: A Role-Aware Audit of WikiBigEdit</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Edison</namePart>
        <namePart type="family">Yang</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Neel</namePart>
        <namePart type="family">Marripalapu</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Akshaj</namePart>
        <namePart type="family">Agadi</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2026-10</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the 11th Workshop on Automated Knowledge Base Construction (AKBC 2026)</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Jan-Christoph</namePart>
            <namePart type="family">Kalo</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Ndapa</namePart>
            <namePart type="family">Nakashole</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Simon</namePart>
            <namePart type="family">Razniewski</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Fabian</namePart>
            <namePart type="given">M</namePart>
            <namePart type="family">Suchanek</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Andreas</namePart>
            <namePart type="family">Vlachos</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Andrew</namePart>
            <namePart type="family">McCallum</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Budapest, Hungary</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
        <identifier type="isbn">979-8-89176-464-4</identifier>
    </relatedItem>
    <abstract>Lifelong knowledge-editing benchmarks turn changes in a structured knowledge base into natural-language edits and evaluation questions. Their value depends on that conversion preserving what each fact says and which entity it concerns. We audit WikiBigEdit, a benchmark of more than 500,000 Wikidata updates, and trace its released construction pipeline to a string scan over a text rendering of Wikidata rather than a parse of its data model. From a uniform random sample of 1,500 rows, we audit the 1,458 resolvable rows against their source revisions and find three consequences: qualifier and reference values are promoted to main facts; 23.1% of resolvable rows attach correct answer text to the wrong entity identifier; and locality probes are chosen by surface-string similarity rather than semantic relatedness. Overall, 22.8% of resolvable rows carry unusable text supervision. In a matched reconstruction that holds the relation and question template fixed, similarity-selected probes show 7–12 percentage points more edit-target leakage, that is, the edited answer surfaces in a neighboring question. This effect persists under parameter editing with ROME, while the aggregate locality score remains unchanged. WikiBigEdit’s official metric does not fully register the leakage. We validate the audit with an independent parser and release the audit code and a provenance-bearing repair set. Our results do not show that published scores are incorrect; they identify structural errors those scores were not designed to detect.</abstract>
    <identifier type="citekey">yang-etal-2026-string</identifier>
    <location>
        <url>https://aclanthology.org/2026.akbc-1.11/</url>
    </location>
    <part>
        <date>2026-10</date>
        <extent unit="page">
            <start>139</start>
            <end>150</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T When String Matching Stands In for a Knowledge Base: A Role-Aware Audit of WikiBigEdit
%A Yang, Edison
%A Marripalapu, Neel
%A Agadi, Akshaj
%Y Kalo, Jan-Christoph
%Y Nakashole, Ndapa
%Y Razniewski, Simon
%Y Suchanek, Fabian M.
%Y Vlachos, Andreas
%Y McCallum, Andrew
%S Proceedings of the 11th Workshop on Automated Knowledge Base Construction (AKBC 2026)
%D 2026
%8 October
%I Association for Computational Linguistics
%C Budapest, Hungary
%@ 979-8-89176-464-4
%F yang-etal-2026-string
%X Lifelong knowledge-editing benchmarks turn changes in a structured knowledge base into natural-language edits and evaluation questions. Their value depends on that conversion preserving what each fact says and which entity it concerns. We audit WikiBigEdit, a benchmark of more than 500,000 Wikidata updates, and trace its released construction pipeline to a string scan over a text rendering of Wikidata rather than a parse of its data model. From a uniform random sample of 1,500 rows, we audit the 1,458 resolvable rows against their source revisions and find three consequences: qualifier and reference values are promoted to main facts; 23.1% of resolvable rows attach correct answer text to the wrong entity identifier; and locality probes are chosen by surface-string similarity rather than semantic relatedness. Overall, 22.8% of resolvable rows carry unusable text supervision. In a matched reconstruction that holds the relation and question template fixed, similarity-selected probes show 7–12 percentage points more edit-target leakage, that is, the edited answer surfaces in a neighboring question. This effect persists under parameter editing with ROME, while the aggregate locality score remains unchanged. WikiBigEdit’s official metric does not fully register the leakage. We validate the audit with an independent parser and release the audit code and a provenance-bearing repair set. Our results do not show that published scores are incorrect; they identify structural errors those scores were not designed to detect.
%U https://aclanthology.org/2026.akbc-1.11/
%P 139-150

Markdown (Informal)

[When String Matching Stands In for a Knowledge Base: A Role-Aware Audit of WikiBigEdit](https://aclanthology.org/2026.akbc-1.11/) (Yang et al., AKBC 2026)

• When String Matching Stands In for a Knowledge Base: A Role-Aware Audit of WikiBigEdit (Yang et al., AKBC 2026)

ACL

• Edison Yang, Neel Marripalapu, and Akshaj Agadi. 2026. When String Matching Stands In for a Knowledge Base: A Role-Aware Audit of WikiBigEdit. In Proceedings of the 11th Workshop on Automated Knowledge Base Construction (AKBC 2026), pages 139–150, Budapest, Hungary. Association for Computational Linguistics.