<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="li-etal-2025-llms-cant">
    <titleInfo>
        <title>When LLMs Can’t Help: Real-World Evaluation of LLMs in Nutrition</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Karen</namePart>
        <namePart type="given">Jia-Hui</namePart>
        <namePart type="family">Li</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Simone</namePart>
        <namePart type="family">Balloccu</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Ondrej</namePart>
        <namePart type="family">Dusek</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Ehud</namePart>
        <namePart type="family">Reiter</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2025-10</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the 18th International Natural Language Generation Conference</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Lucie</namePart>
            <namePart type="family">Flek</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Shashi</namePart>
            <namePart type="family">Narayan</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Lê</namePart>
            <namePart type="given">Hồng</namePart>
            <namePart type="family">Phương</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Jiahuan</namePart>
            <namePart type="family">Pei</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Hanoi, Vietnam</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>The increasing trust in large language models (LLMs), especially in the form of chatbots, is often undermined by the lack of their extrinsic evaluation. This holds particularly true in nutrition, where randomised controlled trials (RCTs) are the gold standard, and experts demand them for evidence-based deployment. LLMs have shown promising results in this field, but these are limited to intrinsic setups. We address this gap by running the first RCT involving LLMs for nutrition. We augment a rule-based chatbot with two LLM-based features: (1) message rephrasing for conversational variety and engagement, and (2) nutritional counselling through a fine-tuned model. In our seven-week RCT (n=81), we compare chatbot variants with and without LLM integration. We measure effects on dietary outcome, emotional well-being, and engagement. Despite our LLM-based features performing well in intrinsic evaluation, we find that they did not yield consistent benefits in real-world deployment. These results highlight critical gaps between intrinsic evaluations and real-world impact, emphasising the need for interdisciplinary, human-centred approaches.</abstract>
    <identifier type="citekey">li-etal-2025-llms-cant</identifier>
    <location>
        <url>https://aclanthology.org/2025.inlg-main.44/</url>
    </location>
    <part>
        <date>2025-10</date>
        <extent unit="page">
            <start>753</start>
            <end>779</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T When LLMs Can’t Help: Real-World Evaluation of LLMs in Nutrition
%A Li, Karen Jia-Hui
%A Balloccu, Simone
%A Dusek, Ondrej
%A Reiter, Ehud
%Y Flek, Lucie
%Y Narayan, Shashi
%Y Phương, Lê Hồng
%Y Pei, Jiahuan
%S Proceedings of the 18th International Natural Language Generation Conference
%D 2025
%8 October
%I Association for Computational Linguistics
%C Hanoi, Vietnam
%F li-etal-2025-llms-cant
%X The increasing trust in large language models (LLMs), especially in the form of chatbots, is often undermined by the lack of their extrinsic evaluation. This holds particularly true in nutrition, where randomised controlled trials (RCTs) are the gold standard, and experts demand them for evidence-based deployment. LLMs have shown promising results in this field, but these are limited to intrinsic setups. We address this gap by running the first RCT involving LLMs for nutrition. We augment a rule-based chatbot with two LLM-based features: (1) message rephrasing for conversational variety and engagement, and (2) nutritional counselling through a fine-tuned model. In our seven-week RCT (n=81), we compare chatbot variants with and without LLM integration. We measure effects on dietary outcome, emotional well-being, and engagement. Despite our LLM-based features performing well in intrinsic evaluation, we find that they did not yield consistent benefits in real-world deployment. These results highlight critical gaps between intrinsic evaluations and real-world impact, emphasising the need for interdisciplinary, human-centred approaches.
%U https://aclanthology.org/2025.inlg-main.44/
%P 753-779

Markdown (Informal)

[When LLMs Can’t Help: Real-World Evaluation of LLMs in Nutrition](https://aclanthology.org/2025.inlg-main.44/) (Li et al., INLG 2025)

• When LLMs Can’t Help: Real-World Evaluation of LLMs in Nutrition (Li et al., INLG 2025)

ACL

• Karen Jia-Hui Li, Simone Balloccu, Ondrej Dusek, and Ehud Reiter. 2025. When LLMs Can’t Help: Real-World Evaluation of LLMs in Nutrition. In Proceedings of the 18th International Natural Language Generation Conference, pages 753–779, Hanoi, Vietnam. Association for Computational Linguistics.