<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="frew-etal-2026-fhirpath">
    <titleInfo>
        <title>FHIRPath-QA: Executable Question Answering over FHIR Electronic Health Records</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Michael</namePart>
        <namePart type="family">Frew</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Nishit</namePart>
        <namePart type="family">Bheda</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Bryan</namePart>
        <namePart type="family">Tripp</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2026-05</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the Third Workshop on Patient-Oriented Language Processing (CL4Health) @ LREC 2026</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Deepak</namePart>
            <namePart type="family">Gupta</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Paul</namePart>
            <namePart type="family">Thompson</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Sophia</namePart>
            <namePart type="family">Ananiadou</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Dina</namePart>
            <namePart type="family">Demner-Fushman</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>ELRA Language Resources Association (ELRA)</publisher>
            <place>
                <placeTerm type="text">Palma, Mallorca (Spain)</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>Though patients are increasingly granted digital access to their electronic health records (EHRs), existing interfaces may not support precise, trustworthy answers to patient-specific questions. Large language models (LLM) show promise in clinical question answering (QA), but retrieval-based approaches are computationally inefficient, prone to hallucination, and difficult to deploy over real-life EHRs. This work introduces FHIRPath-QA, the first open dataset and benchmark for patient-specific QA that includes open-standard FHIRPath queries over real-world clinical data. A text-to-FHIRPath QA paradigm is proposed that shifts reasoning from free-text generation to FHIRPath query synthesis. For o4-mini, this reduced average token usage by 391\times relative to retrieval-first prompting (629,829 vs 1,609 tokens per question) and lowered failure rates from 0.36 to 0.09 on clinician-phrased questions. Built on MIMIC-IV on FHIR Demo, the dataset pairs over 14k natural language questions in patient and clinician phrasing with validated FHIRPath queries and answers. Empirically, the evaluated LLMs achieve at most 42% accuracy, highlighting the challenge of the task, but benefit strongly from supervised fine-tuning, with query synthesis accuracy improving from 27% to 79% for 4o-mini. These results highlight that text-to-FHIRPath synthesis has the potential to serve as a practical foundation for safe, efficient, and interoperable consumer health applications, and the FHIRPath-QA dataset and benchmark serve as a starting point for future research on the topic. The full dataset and generation code can be accessed on \hrefhttps://github.com/mooshifrew/fhirpath-qaGitHub.</abstract>
    <identifier type="citekey">frew-etal-2026-fhirpath</identifier>
    <identifier type="doi">10.63317/5q8qi6jqnz56</identifier>
    <location>
        <url>https://aclanthology.org/2026.cl4health-1.1/</url>
    </location>
    <part>
        <date>2026-05</date>
        <extent unit="page">
            <start>1</start>
            <end>14</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T FHIRPath-QA: Executable Question Answering over FHIR Electronic Health Records
%A Frew, Michael
%A Bheda, Nishit
%A Tripp, Bryan
%Y Gupta, Deepak
%Y Thompson, Paul
%Y Ananiadou, Sophia
%Y Demner-Fushman, Dina
%S Proceedings of the Third Workshop on Patient-Oriented Language Processing (CL4Health) @ LREC 2026
%D 2026
%8 May
%I ELRA Language Resources Association (ELRA)
%C Palma, Mallorca (Spain)
%F frew-etal-2026-fhirpath
%X Though patients are increasingly granted digital access to their electronic health records (EHRs), existing interfaces may not support precise, trustworthy answers to patient-specific questions. Large language models (LLM) show promise in clinical question answering (QA), but retrieval-based approaches are computationally inefficient, prone to hallucination, and difficult to deploy over real-life EHRs. This work introduces FHIRPath-QA, the first open dataset and benchmark for patient-specific QA that includes open-standard FHIRPath queries over real-world clinical data. A text-to-FHIRPath QA paradigm is proposed that shifts reasoning from free-text generation to FHIRPath query synthesis. For o4-mini, this reduced average token usage by 391\times relative to retrieval-first prompting (629,829 vs 1,609 tokens per question) and lowered failure rates from 0.36 to 0.09 on clinician-phrased questions. Built on MIMIC-IV on FHIR Demo, the dataset pairs over 14k natural language questions in patient and clinician phrasing with validated FHIRPath queries and answers. Empirically, the evaluated LLMs achieve at most 42% accuracy, highlighting the challenge of the task, but benefit strongly from supervised fine-tuning, with query synthesis accuracy improving from 27% to 79% for 4o-mini. These results highlight that text-to-FHIRPath synthesis has the potential to serve as a practical foundation for safe, efficient, and interoperable consumer health applications, and the FHIRPath-QA dataset and benchmark serve as a starting point for future research on the topic. The full dataset and generation code can be accessed on \hrefhttps://github.com/mooshifrew/fhirpath-qaGitHub.
%R 10.63317/5q8qi6jqnz56
%U https://aclanthology.org/2026.cl4health-1.1/
%U https://doi.org/10.63317/5q8qi6jqnz56
%P 1-14

Markdown (Informal)

[FHIRPath-QA: Executable Question Answering over FHIR Electronic Health Records](https://aclanthology.org/2026.cl4health-1.1/) (Frew et al., CL4Health 2026)

• FHIRPath-QA: Executable Question Answering over FHIR Electronic Health Records (Frew et al., CL4Health 2026)

ACL

• Michael Frew, Nishit Bheda, and Bryan Tripp. 2026. FHIRPath-QA: Executable Question Answering over FHIR Electronic Health Records. In Proceedings of the Third Workshop on Patient-Oriented Language Processing (CL4Health) @ LREC 2026, pages 1–14, Palma, Mallorca (Spain). ELRA Language Resources Association (ELRA).