<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="kim-etal-2026-learning">
    <titleInfo>
        <title>Learning to Combine AI Annotations for Improved Biomedical Relevance Labeling</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Won</namePart>
        <namePart type="given">Gyu</namePart>
        <namePart type="family">Kim</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Lana</namePart>
        <namePart type="family">Yeganova</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Shubo</namePart>
        <namePart type="family">Tian</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Donald</namePart>
        <namePart type="family">Comeau</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">W</namePart>
        <namePart type="given">John</namePart>
        <namePart type="family">Wilbur</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Zhiyong</namePart>
        <namePart type="family">Lu</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2026-07</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>BioNLP 2026</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Dina</namePart>
            <namePart type="family">Demner-Fushman</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Sophia</namePart>
            <namePart type="family">Ananiadou</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Kirk</namePart>
            <namePart type="family">Roberts</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Junichi</namePart>
            <namePart type="family">Tsujii</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">San Diego, California</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
        <identifier type="isbn">979-8-89176-434-7</identifier>
    </relatedItem>
    <abstract>Accurate labeling of relevance between biomedical abstracts is essential for improving information retrieval, semantic similarity modeling, training of ranking systems and other Natural Language Processing tasks. However, manual annotations are time-consuming, labor intensive and costly. Studies show that large language models (LLMs) can facilitate automated annotation, but their performance still falls short of human expert-level accuracy, especially in domain-specific tasks. It has been shown that combining annotations from multiple non-expert annotators can achieve performance comparable to, or even exceeding, that of trained experts. Based on this evidence, we treat AI-generated annotations as contributions from non-expert annotators and combine them using Learning to Rank framework. Our results show significant improvement in overall annotation quality. The proposed method looks promising to reduce reliance on human annotation while maintaining reliable performance for large-scale biomedical applications.</abstract>
    <identifier type="citekey">kim-etal-2026-learning</identifier>
    <location>
        <url>https://aclanthology.org/2026.bionlp-1.40/</url>
    </location>
    <part>
        <date>2026-07</date>
        <extent unit="page">
            <start>502</start>
            <end>507</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Learning to Combine AI Annotations for Improved Biomedical Relevance Labeling
%A Kim, Won Gyu
%A Yeganova, Lana
%A Tian, Shubo
%A Comeau, Donald
%A Wilbur, W. John
%A Lu, Zhiyong
%Y Demner-Fushman, Dina
%Y Ananiadou, Sophia
%Y Roberts, Kirk
%Y Tsujii, Junichi
%S BioNLP 2026
%D 2026
%8 July
%I Association for Computational Linguistics
%C San Diego, California
%@ 979-8-89176-434-7
%F kim-etal-2026-learning
%X Accurate labeling of relevance between biomedical abstracts is essential for improving information retrieval, semantic similarity modeling, training of ranking systems and other Natural Language Processing tasks. However, manual annotations are time-consuming, labor intensive and costly. Studies show that large language models (LLMs) can facilitate automated annotation, but their performance still falls short of human expert-level accuracy, especially in domain-specific tasks. It has been shown that combining annotations from multiple non-expert annotators can achieve performance comparable to, or even exceeding, that of trained experts. Based on this evidence, we treat AI-generated annotations as contributions from non-expert annotators and combine them using Learning to Rank framework. Our results show significant improvement in overall annotation quality. The proposed method looks promising to reduce reliance on human annotation while maintaining reliable performance for large-scale biomedical applications.
%U https://aclanthology.org/2026.bionlp-1.40/
%P 502-507

Markdown (Informal)

[Learning to Combine AI Annotations for Improved Biomedical Relevance Labeling](https://aclanthology.org/2026.bionlp-1.40/) (Kim et al., BioNLP 2026)

• Learning to Combine AI Annotations for Improved Biomedical Relevance Labeling (Kim et al., BioNLP 2026)

ACL

• Won Gyu Kim, Lana Yeganova, Shubo Tian, Donald Comeau, W John Wilbur, and Zhiyong Lu. 2026. Learning to Combine AI Annotations for Improved Biomedical Relevance Labeling. In BioNLP 2026, pages 502–507, San Diego, California. Association for Computational Linguistics.