<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="sohrab-2026-biocat5">
    <titleInfo>
        <title>BioCAT5: Concept Mention-Aware Text-to-Text Transfer Transformer for Biomedical Literature</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Mohammad</namePart>
        <namePart type="given">Golam</namePart>
        <namePart type="family">Sohrab</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2026-11</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Findings of the Association for Computational Linguistics: IJCNLP-AACL 2026</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Yue</namePart>
            <namePart type="family">Zhang</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Ruifeng</namePart>
            <namePart type="family">Xu</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Ayu</namePart>
            <namePart type="family">Purwarianti</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Tong</namePart>
            <namePart type="family">Rong</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Jun</namePart>
            <namePart type="family">Suzuki</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Hengqin, China</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
        <identifier type="isbn">979-8-89176-444-6</identifier>
    </relatedItem>
    <abstract>Integrating biomedical knowledge information like concept name a.k.a concept mention during transfer learning is not a common fashion as it needs an alignment of large-scale concepts with unstructured biomedical text during pre-training. Existing learning objectives on biomedical text mainly rely on masked language modeling or random span denoising where grafting knowledge like integrating medical concepts has never been explored nor been discussed to facilitate the model pre-training. A prevailing assumption is that such concept-aware pre-training on biomedical text can be effective in biomedical task-specific language models. Based on this assumption, we introduce Bio-CAT5, a biomedical domain-specific text-to-text transfer transformer that integrates concept related knowledge from the unified medical language system (UMLS) during pre-training. BioCAT5 leverages concept-aware denoising– an out-of-the-box language model learning objective that is trained to reconstruct text by masking out the concept spans of the input sequence. To facilitate the BioCAT5 pre-training, large-scale concept mentions from UMLS are considered to align with PubMed abstract text to create the pre-training dataset. A publicly available comprehensive biomedical natural language processing (NLP) benchmark datasets including named entity recognition (NER), relation extraction, question answering are evaluated. Besides, the general language understanding evaluation (GLUE) benchmark datasets are considered to show the adaptability of BioCAT5 in general domain tasks. We conduct a series of experiments with ablation studies to validate our proposed approach. The experiment results show that the alignment of concepts during pre-training is potentially significant for biomedical NLP tasks, especially on the NER datasets. Our code is publicly released on GitHub (Available at: https://github.com/aistairc/biocat5) under the Apache 2.0 License.</abstract>
    <identifier type="citekey">sohrab-2026-biocat5</identifier>
    <location>
        <url>https://aclanthology.org/2026.findings-aacl.73/</url>
    </location>
    <part>
        <date>2026-11</date>
        <extent unit="page">
            <start>1425</start>
            <end>1439</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T BioCAT5: Concept Mention-Aware Text-to-Text Transfer Transformer for Biomedical Literature
%A Sohrab, Mohammad Golam
%Y Zhang, Yue
%Y Xu, Ruifeng
%Y Purwarianti, Ayu
%Y Rong, Tong
%Y Suzuki, Jun
%S Findings of the Association for Computational Linguistics: IJCNLP-AACL 2026
%D 2026
%8 November
%I Association for Computational Linguistics
%C Hengqin, China
%@ 979-8-89176-444-6
%F sohrab-2026-biocat5
%X Integrating biomedical knowledge information like concept name a.k.a concept mention during transfer learning is not a common fashion as it needs an alignment of large-scale concepts with unstructured biomedical text during pre-training. Existing learning objectives on biomedical text mainly rely on masked language modeling or random span denoising where grafting knowledge like integrating medical concepts has never been explored nor been discussed to facilitate the model pre-training. A prevailing assumption is that such concept-aware pre-training on biomedical text can be effective in biomedical task-specific language models. Based on this assumption, we introduce Bio-CAT5, a biomedical domain-specific text-to-text transfer transformer that integrates concept related knowledge from the unified medical language system (UMLS) during pre-training. BioCAT5 leverages concept-aware denoising– an out-of-the-box language model learning objective that is trained to reconstruct text by masking out the concept spans of the input sequence. To facilitate the BioCAT5 pre-training, large-scale concept mentions from UMLS are considered to align with PubMed abstract text to create the pre-training dataset. A publicly available comprehensive biomedical natural language processing (NLP) benchmark datasets including named entity recognition (NER), relation extraction, question answering are evaluated. Besides, the general language understanding evaluation (GLUE) benchmark datasets are considered to show the adaptability of BioCAT5 in general domain tasks. We conduct a series of experiments with ablation studies to validate our proposed approach. The experiment results show that the alignment of concepts during pre-training is potentially significant for biomedical NLP tasks, especially on the NER datasets. Our code is publicly released on GitHub (Available at: https://github.com/aistairc/biocat5) under the Apache 2.0 License.
%U https://aclanthology.org/2026.findings-aacl.73/
%P 1425-1439

Markdown (Informal)

[BioCAT5: Concept Mention-Aware Text-to-Text Transfer Transformer for Biomedical Literature](https://aclanthology.org/2026.findings-aacl.73/) (Sohrab, Findings 2026)

• BioCAT5: Concept Mention-Aware Text-to-Text Transfer Transformer for Biomedical Literature (Sohrab, Findings 2026)

ACL

• Mohammad Golam Sohrab. 2026. BioCAT5: Concept Mention-Aware Text-to-Text Transfer Transformer for Biomedical Literature. In Findings of the Association for Computational Linguistics: IJCNLP-AACL 2026, pages 1425–1439, Hengqin, China. Association for Computational Linguistics.