<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="ali-2026-clinical">
    <titleInfo>
        <title>A Clinical SKOS Ontology and Evaluation Benchmark for LLM Query Generation over ICU Knowledge Graphs</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Khurrum</namePart>
        <namePart type="family">Ali</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2026-05</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the Knowledge Graphs and Large Language Models Workshop (KG-LLM) @ LREC26</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Gilles</namePart>
            <namePart type="family">Sérasset</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Katerina</namePart>
            <namePart type="family">Gkirtzou</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Michael</namePart>
            <namePart type="family">Cochez</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Jan-Christoph</namePart>
            <namePart type="family">Kalo</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>ELRA Language Resources Association (ELRA)</publisher>
            <place>
                <placeTerm type="text">Palma, Mallorca (Spain)</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>Whencliniciansquerydatabasesusingeverydaylanguage—“codebluepatients” or”sugardisease”—LargeLanguage Models must bridge a lexical gap between colloquial speech and formal clinical terminology. While highly capable cloudmodelscanleverageexternalontologieslike SKOStoresolvethese termsviaSPARQLqueries, hospital privacy regulations often mandate the use of air-gapped local LLMs (4–8B parameters). We evaluate query generation across scales (Gemini 2.0 Flash vs. LLaMA 3.1 8B) using ClinSKOS-ICU, a curated ontology of 421 ICU concepts, and ClinNLU, an evaluation benchmark. We identify a critical “Privacy Penalty”: while Gemini achieves 90.2% ontology deferral under an RDF+SKOS architecture, local LLMs exhibit a 100% “Semantic Bypass” vulnerability, hardcodingformaltermsintoqueriesratherthandeferringtothegraph. ToimprovelocalLLMgrounding, weintroduce Architectural Decomposition, a pipeline that restricts the LLM to Grammar-Constrained JSON entity extraction and delegates query generation to deterministic code. This structural pivot entirely eliminates Semantic Bypass (0%) and achieves an 80.4% ontology deferral rate on an 8B model, suggesting that decoupled extraction is highly effective for enforcing W3C semantic compliance on privacy-preserving local hardware.</abstract>
    <identifier type="citekey">ali-2026-clinical</identifier>
    <identifier type="doi">10.63317/4ph5haswh86k</identifier>
    <location>
        <url>https://aclanthology.org/2026.kallm-1.9/</url>
    </location>
    <part>
        <date>2026-05</date>
        <extent unit="page">
            <start>82</start>
            <end>92</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T A Clinical SKOS Ontology and Evaluation Benchmark for LLM Query Generation over ICU Knowledge Graphs
%A Ali, Khurrum
%Y Sérasset, Gilles
%Y Gkirtzou, Katerina
%Y Cochez, Michael
%Y Kalo, Jan-Christoph
%S Proceedings of the Knowledge Graphs and Large Language Models Workshop (KG-LLM) @ LREC26
%D 2026
%8 May
%I ELRA Language Resources Association (ELRA)
%C Palma, Mallorca (Spain)
%F ali-2026-clinical
%X Whencliniciansquerydatabasesusingeverydaylanguage—“codebluepatients” or”sugardisease”—LargeLanguage Models must bridge a lexical gap between colloquial speech and formal clinical terminology. While highly capable cloudmodelscanleverageexternalontologieslike SKOStoresolvethese termsviaSPARQLqueries, hospital privacy regulations often mandate the use of air-gapped local LLMs (4–8B parameters). We evaluate query generation across scales (Gemini 2.0 Flash vs. LLaMA 3.1 8B) using ClinSKOS-ICU, a curated ontology of 421 ICU concepts, and ClinNLU, an evaluation benchmark. We identify a critical “Privacy Penalty”: while Gemini achieves 90.2% ontology deferral under an RDF+SKOS architecture, local LLMs exhibit a 100% “Semantic Bypass” vulnerability, hardcodingformaltermsintoqueriesratherthandeferringtothegraph. ToimprovelocalLLMgrounding, weintroduce Architectural Decomposition, a pipeline that restricts the LLM to Grammar-Constrained JSON entity extraction and delegates query generation to deterministic code. This structural pivot entirely eliminates Semantic Bypass (0%) and achieves an 80.4% ontology deferral rate on an 8B model, suggesting that decoupled extraction is highly effective for enforcing W3C semantic compliance on privacy-preserving local hardware.
%R 10.63317/4ph5haswh86k
%U https://aclanthology.org/2026.kallm-1.9/
%U https://doi.org/10.63317/4ph5haswh86k
%P 82-92

Markdown (Informal)

[A Clinical SKOS Ontology and Evaluation Benchmark for LLM Query Generation over ICU Knowledge Graphs](https://aclanthology.org/2026.kallm-1.9/) (Ali, KaLLM 2026)

• A Clinical SKOS Ontology and Evaluation Benchmark for LLM Query Generation over ICU Knowledge Graphs (Ali, KaLLM 2026)

ACL

• Khurrum Ali. 2026. A Clinical SKOS Ontology and Evaluation Benchmark for LLM Query Generation over ICU Knowledge Graphs. In Proceedings of the Knowledge Graphs and Large Language Models Workshop (KG-LLM) @ LREC26, pages 82–92, Palma, Mallorca (Spain). ELRA Language Resources Association (ELRA).