<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="thompson-etal-2026-evaluating">
    <titleInfo>
        <title>Evaluating Professional Acceptability of LLM-Generated Systematic Review Summaries in Healthcare: Psychiatrists’ Perspectives</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Paul</namePart>
        <namePart type="family">Thompson</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Artemis</namePart>
        <namePart type="family">Boulogeorgou</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Fotini</namePart>
        <namePart type="family">Kaponi</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Efstathia</namePart>
        <namePart type="family">Soufleri</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Sophia</namePart>
        <namePart type="family">Ananiadou</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2026-05</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the Third Workshop on Patient-Oriented Language Processing (CL4Health) @ LREC 2026</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Deepak</namePart>
            <namePart type="family">Gupta</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Paul</namePart>
            <namePart type="family">Thompson</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Sophia</namePart>
            <namePart type="family">Ananiadou</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Dina</namePart>
            <namePart type="family">Demner-Fushman</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>ELRA Language Resources Association (ELRA)</publisher>
            <place>
                <placeTerm type="text">Palma, Mallorca (Spain)</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>Cochrane systematic reviews evaluate the effectiveness and safety of medical interventions. Patients can benefit from clinicians’ integration of outcomes of these reviews into their daily practices. However, systematic reviews are usually long documents; even their abstracts can extend to 1000 words, making rapid appraisal challenging for busy health professionals. Large language models (LLMs) offer potential to further distil these abstracts. Nevertheless, generating high-quality, clinician-oriented summaries in this context is non-trivial. They must comprehensively cover the original abstract, while remaining accurate and professionally acceptable, i.e., retaining all clinically important details. To address this challenge, we have developed a novel dataset, PsycSumEval, comprising summaries generated by four different LLMs for 115 Cochrane abstracts concerning mental health. Psychiatrists evaluated each summary across nine content dimensions, assigning scores and providing free-text justifications that highlight inaccuracies and missing details. The corpus provides fine-grained insight into how psychiatrists assess professional acceptability of compressed medical evidence. Rather than treating agreement as a merely statistical endpoint, we capture structured expert judgments alongside their rationales, enabling transparent analysis of where professional norms are stable and where interpretive latitude persists. We contribute both a rigorous evaluation dataset and an explicit model of expert acceptability criteria for medical evidence summarisation.</abstract>
    <identifier type="citekey">thompson-etal-2026-evaluating</identifier>
    <identifier type="doi">10.63317/4exjqeuaw9ti</identifier>
    <location>
        <url>https://aclanthology.org/2026.cl4health-1.18/</url>
    </location>
    <part>
        <date>2026-05</date>
        <extent unit="page">
            <start>190</start>
            <end>206</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Evaluating Professional Acceptability of LLM-Generated Systematic Review Summaries in Healthcare: Psychiatrists’ Perspectives
%A Thompson, Paul
%A Boulogeorgou, Artemis
%A Kaponi, Fotini
%A Soufleri, Efstathia
%A Ananiadou, Sophia
%Y Gupta, Deepak
%Y Thompson, Paul
%Y Ananiadou, Sophia
%Y Demner-Fushman, Dina
%S Proceedings of the Third Workshop on Patient-Oriented Language Processing (CL4Health) @ LREC 2026
%D 2026
%8 May
%I ELRA Language Resources Association (ELRA)
%C Palma, Mallorca (Spain)
%F thompson-etal-2026-evaluating
%X Cochrane systematic reviews evaluate the effectiveness and safety of medical interventions. Patients can benefit from clinicians’ integration of outcomes of these reviews into their daily practices. However, systematic reviews are usually long documents; even their abstracts can extend to 1000 words, making rapid appraisal challenging for busy health professionals. Large language models (LLMs) offer potential to further distil these abstracts. Nevertheless, generating high-quality, clinician-oriented summaries in this context is non-trivial. They must comprehensively cover the original abstract, while remaining accurate and professionally acceptable, i.e., retaining all clinically important details. To address this challenge, we have developed a novel dataset, PsycSumEval, comprising summaries generated by four different LLMs for 115 Cochrane abstracts concerning mental health. Psychiatrists evaluated each summary across nine content dimensions, assigning scores and providing free-text justifications that highlight inaccuracies and missing details. The corpus provides fine-grained insight into how psychiatrists assess professional acceptability of compressed medical evidence. Rather than treating agreement as a merely statistical endpoint, we capture structured expert judgments alongside their rationales, enabling transparent analysis of where professional norms are stable and where interpretive latitude persists. We contribute both a rigorous evaluation dataset and an explicit model of expert acceptability criteria for medical evidence summarisation.
%R 10.63317/4exjqeuaw9ti
%U https://aclanthology.org/2026.cl4health-1.18/
%U https://doi.org/10.63317/4exjqeuaw9ti
%P 190-206

Markdown (Informal)

[Evaluating Professional Acceptability of LLM-Generated Systematic Review Summaries in Healthcare: Psychiatrists’ Perspectives](https://aclanthology.org/2026.cl4health-1.18/) (Thompson et al., CL4Health 2026)

• Evaluating Professional Acceptability of LLM-Generated Systematic Review Summaries in Healthcare: Psychiatrists’ Perspectives (Thompson et al., CL4Health 2026)

ACL

• Paul Thompson, Artemis Boulogeorgou, Fotini Kaponi, Efstathia Soufleri, and Sophia Ananiadou. 2026. Evaluating Professional Acceptability of LLM-Generated Systematic Review Summaries in Healthcare: Psychiatrists’ Perspectives. In Proceedings of the Third Workshop on Patient-Oriented Language Processing (CL4Health) @ LREC 2026, pages 190–206, Palma, Mallorca (Spain). ELRA Language Resources Association (ELRA).