<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="chakar-etal-2026-benchmark">
    <titleInfo>
        <title>A Benchmark for Overgeneration Detection in Biomedical Text Simplification</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Berkay</namePart>
        <namePart type="family">Chakar</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Liana</namePart>
        <namePart type="family">Ermakova</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Jaap</namePart>
        <namePart type="family">Kamps</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2026-05</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the 2nd Workshop on Evaluating Text Difficulty in a Multilingual Context (DeTermIt! 2026)</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Giorgio</namePart>
            <namePart type="given">Maria</namePart>
            <namePart type="family">Di Nunzio</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Federica</namePart>
            <namePart type="family">Vezzani</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Liana</namePart>
            <namePart type="family">Ermakova</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Hosein</namePart>
            <namePart type="family">Azarbonyad</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Jaap</namePart>
            <namePart type="family">Kamps</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>ELRA Language Resources Association (ELRA)</publisher>
            <place>
                <placeTerm type="text">Palma, Mallorca (Spain)</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>Large Language Models deployed for biomedical text simplification frequently produce overgeneration: extraneous content appended beyond the faithful simplification, including leaked model instructions, ungrounded medical claims, and repetitive text. Despite its prevalence, this failure mode remains largely unaddressed. We present a benchmark for document-level overgeneration detection, releasing two resources: SimpleOG-manual, 500 abstract-level examples with human-validated positive labels, and SimpleOG-auto, over 46,000 automatically labeled abstract-level examples derived from submissions to the CLEF 2025 SimpleText Track. Our method exploits the positional regularity of overgeneration in simplification output through sequence alignment, identifying trailing content that lacks a corresponding segment in the source. Human validation of 117 automatically flagged positives confirms \ensuremath\sim95% precision, with leaked model instructions accounting for 75.7% of confirmed cases. Analysis across teams and models reveals that overgeneration is primarily driven by system-level choices, such as prompting and post-processing, rather than by model architecture. We evaluate three detection paradigms and find that sentence similarity (F1 = 0.731, ROC-AUC = 0.915) surprisingly outperforms both NLI-based and LLM-based approaches, suggesting that overgenerated content occupies distinct semantic regions from source material.</abstract>
    <identifier type="citekey">chakar-etal-2026-benchmark</identifier>
    <identifier type="doi">10.63317/3aodve3ow6a7</identifier>
    <location>
        <url>https://aclanthology.org/2026.determit-1.2/</url>
    </location>
    <part>
        <date>2026-05</date>
        <extent unit="page">
            <start>12</start>
            <end>21</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T A Benchmark for Overgeneration Detection in Biomedical Text Simplification
%A Chakar, Berkay
%A Ermakova, Liana
%A Kamps, Jaap
%Y Di Nunzio, Giorgio Maria
%Y Vezzani, Federica
%Y Ermakova, Liana
%Y Azarbonyad, Hosein
%Y Kamps, Jaap
%S Proceedings of the 2nd Workshop on Evaluating Text Difficulty in a Multilingual Context (DeTermIt! 2026)
%D 2026
%8 May
%I ELRA Language Resources Association (ELRA)
%C Palma, Mallorca (Spain)
%F chakar-etal-2026-benchmark
%X Large Language Models deployed for biomedical text simplification frequently produce overgeneration: extraneous content appended beyond the faithful simplification, including leaked model instructions, ungrounded medical claims, and repetitive text. Despite its prevalence, this failure mode remains largely unaddressed. We present a benchmark for document-level overgeneration detection, releasing two resources: SimpleOG-manual, 500 abstract-level examples with human-validated positive labels, and SimpleOG-auto, over 46,000 automatically labeled abstract-level examples derived from submissions to the CLEF 2025 SimpleText Track. Our method exploits the positional regularity of overgeneration in simplification output through sequence alignment, identifying trailing content that lacks a corresponding segment in the source. Human validation of 117 automatically flagged positives confirms \ensuremath\sim95% precision, with leaked model instructions accounting for 75.7% of confirmed cases. Analysis across teams and models reveals that overgeneration is primarily driven by system-level choices, such as prompting and post-processing, rather than by model architecture. We evaluate three detection paradigms and find that sentence similarity (F1 = 0.731, ROC-AUC = 0.915) surprisingly outperforms both NLI-based and LLM-based approaches, suggesting that overgenerated content occupies distinct semantic regions from source material.
%R 10.63317/3aodve3ow6a7
%U https://aclanthology.org/2026.determit-1.2/
%U https://doi.org/10.63317/3aodve3ow6a7
%P 12-21

Markdown (Informal)

[A Benchmark for Overgeneration Detection in Biomedical Text Simplification](https://aclanthology.org/2026.determit-1.2/) (Chakar et al., DeTermIt 2026)

• A Benchmark for Overgeneration Detection in Biomedical Text Simplification (Chakar et al., DeTermIt 2026)

ACL

• Berkay Chakar, Liana Ermakova, and Jaap Kamps. 2026. A Benchmark for Overgeneration Detection in Biomedical Text Simplification. In Proceedings of the 2nd Workshop on Evaluating Text Difficulty in a Multilingual Context (DeTermIt! 2026), pages 12–21, Palma, Mallorca (Spain). ELRA Language Resources Association (ELRA).