<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="rajwal-etal-2026-diagnostic">
    <titleInfo>
        <title>Diagnostic Reasoning with Large Language Models for a Rare Disease: Case Study of Primary Ciliary Dyskinesia</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Swati</namePart>
        <namePart type="family">Rajwal</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Mary</namePart>
        <namePart type="given">Ellen</namePart>
        <namePart type="given">M</namePart>
        <namePart type="family">Fain</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Lokesh</namePart>
        <namePart type="family">Guglani</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Abeed</namePart>
        <namePart type="family">Sarker</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2026-05</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the Third Workshop on Patient-Oriented Language Processing (CL4Health) @ LREC 2026</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Deepak</namePart>
            <namePart type="family">Gupta</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Paul</namePart>
            <namePart type="family">Thompson</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Sophia</namePart>
            <namePart type="family">Ananiadou</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Dina</namePart>
            <namePart type="family">Demner-Fushman</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>ELRA Language Resources Association (ELRA)</publisher>
            <place>
                <placeTerm type="text">Palma, Mallorca (Spain)</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>Primary ciliary dyskinesia (PCD) is a rare pediatric lung disease that is frequently underdiagnosed due to nonspecific early symptoms and limited clinical exposure. We investigate whether large language models (LLMs) can support early diagnostic reasoning using real-world pediatric pulmonology notes written before the final diagnosis. We curated 58 de-identified first-visit notes (28 confirmed PCD, 30 controls) and evaluated five open-source LLMs using a standardized zero-shot prompt to produce structured outputs, including PCD evaluation recommendations, justifications, and suggested tests. Quantitative performance was assessed against expert-validated labels using sensitivity, specificity, and accuracy, and a clinician qualitatively reviewed all explanations and testing recommendations for clinical soundness. Sensitivity ranged from 0.48 to 1.00 and specificity from 0.10 to 0.48 (excluding uncertain outputs), with a best accuracy of 0.75. A majority-vote ensemble of five open-source LLMs achieved perfect sensitivity (1.00) with accuracy of 0.73. While models often identified clinically relevant signals in unstructured notes, explanations and testing recommendations were frequently only partially sound. These findings suggest LLMs may serve as cautious early screening aids for rare disease suspicion, but not as standalone diagnostic tools. This work further highlights the need for larger, multi-site evaluation on longitudinal clinical text.</abstract>
    <identifier type="citekey">rajwal-etal-2026-diagnostic</identifier>
    <identifier type="doi">10.63317/3zvnibnkbvsc</identifier>
    <location>
        <url>https://aclanthology.org/2026.cl4health-1.27/</url>
    </location>
    <part>
        <date>2026-05</date>
        <extent unit="page">
            <start>307</start>
            <end>319</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Diagnostic Reasoning with Large Language Models for a Rare Disease: Case Study of Primary Ciliary Dyskinesia
%A Rajwal, Swati
%A Fain, Mary Ellen M.
%A Guglani, Lokesh
%A Sarker, Abeed
%Y Gupta, Deepak
%Y Thompson, Paul
%Y Ananiadou, Sophia
%Y Demner-Fushman, Dina
%S Proceedings of the Third Workshop on Patient-Oriented Language Processing (CL4Health) @ LREC 2026
%D 2026
%8 May
%I ELRA Language Resources Association (ELRA)
%C Palma, Mallorca (Spain)
%F rajwal-etal-2026-diagnostic
%X Primary ciliary dyskinesia (PCD) is a rare pediatric lung disease that is frequently underdiagnosed due to nonspecific early symptoms and limited clinical exposure. We investigate whether large language models (LLMs) can support early diagnostic reasoning using real-world pediatric pulmonology notes written before the final diagnosis. We curated 58 de-identified first-visit notes (28 confirmed PCD, 30 controls) and evaluated five open-source LLMs using a standardized zero-shot prompt to produce structured outputs, including PCD evaluation recommendations, justifications, and suggested tests. Quantitative performance was assessed against expert-validated labels using sensitivity, specificity, and accuracy, and a clinician qualitatively reviewed all explanations and testing recommendations for clinical soundness. Sensitivity ranged from 0.48 to 1.00 and specificity from 0.10 to 0.48 (excluding uncertain outputs), with a best accuracy of 0.75. A majority-vote ensemble of five open-source LLMs achieved perfect sensitivity (1.00) with accuracy of 0.73. While models often identified clinically relevant signals in unstructured notes, explanations and testing recommendations were frequently only partially sound. These findings suggest LLMs may serve as cautious early screening aids for rare disease suspicion, but not as standalone diagnostic tools. This work further highlights the need for larger, multi-site evaluation on longitudinal clinical text.
%R 10.63317/3zvnibnkbvsc
%U https://aclanthology.org/2026.cl4health-1.27/
%U https://doi.org/10.63317/3zvnibnkbvsc
%P 307-319

Markdown (Informal)

[Diagnostic Reasoning with Large Language Models for a Rare Disease: Case Study of Primary Ciliary Dyskinesia](https://aclanthology.org/2026.cl4health-1.27/) (Rajwal et al., CL4Health 2026)

• Diagnostic Reasoning with Large Language Models for a Rare Disease: Case Study of Primary Ciliary Dyskinesia (Rajwal et al., CL4Health 2026)

ACL

• Swati Rajwal, Mary Ellen M. Fain, Lokesh Guglani, and Abeed Sarker. 2026. Diagnostic Reasoning with Large Language Models for a Rare Disease: Case Study of Primary Ciliary Dyskinesia. In Proceedings of the Third Workshop on Patient-Oriented Language Processing (CL4Health) @ LREC 2026, pages 307–319, Palma, Mallorca (Spain). ELRA Language Resources Association (ELRA).