Exploring LLM-Based Assessment of Italian Middle School Writing: A Pilot Study

Adriana Mirabella; Dominique Brunato

doi:10.18653/v1/2025.bea-1.51

Exploring LLM-Based Assessment of Italian Middle School Writing: A Pilot Study

Correct Metadata for

Important: The Anthology treat PDFs as authoritative. Please use this form only to correct data that is out of line with the PDF. See our corrections guidelines if you need to change the PDF.

Title Adjust the title. Retain tags such as <fixed-case>.

Authors Adjust author names and order to match the PDF.

Abstract Correct abstract if needed. Retain XML formatting tags such as <tex-math>.

Verification against PDF Ensure that the new title/authors match the snapshot below. (If there is no snapshot or it is too small, consult the PDF.)

Authors concatenated from the text boxes above:

ALL author names match the snapshot above—including middle initials, hyphens, and accents.

Abstract

This study investigates the use of ChatGPT for Automated Essay Scoring (AES) in assessing Italian middle school students’ written texts. Using rubrics targeting grammar, coherence and argumentation, we compare AI-generated feedback with that of a human teacher on a newly collected corpus of students’ essays. Despite some differences, ChatGPT provided detailed and timely feedback that complements the teacher’s role. These findings underscore the potential of generative AI to improve the assessment of writing, providing useful insights for educators and supporting students in developing their writing skills.

Anthology ID:: 2025.bea-1.51
Volume:: Proceedings of the 20th Workshop on Innovative Use of NLP for Building Educational Applications (BEA 2025)
Month:: July
Year:: 2025
Address:: Vienna, Austria
Editors:: Ekaterina Kochmar, Bashar Alhafni, Marie Bexte, Jill Burstein, Andrea Horbach, Ronja Laarmann-Quante, Anaïs Tack, Victoria Yaneva, Zheng Yuan
Venues:: BEA | WS
SIG:: SIGEDU
Publisher:: Association for Computational Linguistics
Note:
Pages:: 708–715
Language:
URL:: https://aclanthology.org/2025.bea-1.51/
DOI:: 10.18653/v1/2025.bea-1.51
Bibkey:
Cite (ACL):: Adriana Mirabella and Dominique Brunato. 2025. Exploring LLM-Based Assessment of Italian Middle School Writing: A Pilot Study. In Proceedings of the 20th Workshop on Innovative Use of NLP for Building Educational Applications (BEA 2025), pages 708–715, Vienna, Austria. Association for Computational Linguistics.
Cite (Informal):: Exploring LLM-Based Assessment of Italian Middle School Writing: A Pilot Study (Mirabella & Brunato, BEA 2025)
Copy Citation:
PDF:: https://aclanthology.org/2025.bea-1.51.pdf

PDF Cite Search Fix data

Export citation

BibTeX
MODS XML
Endnote
Preformatted

@inproceedings{mirabella-brunato-2025-exploring,
    title = "Exploring {LLM}-Based Assessment of {I}talian Middle School Writing: A Pilot Study",
    author = "Mirabella, Adriana  and
      Brunato, Dominique",
    editor = {Kochmar, Ekaterina  and
      Alhafni, Bashar  and
      Bexte, Marie  and
      Burstein, Jill  and
      Horbach, Andrea  and
      Laarmann-Quante, Ronja  and
      Tack, Ana{\"i}s  and
      Yaneva, Victoria  and
      Yuan, Zheng},
    booktitle = "Proceedings of the 20th Workshop on Innovative Use of NLP for Building Educational Applications (BEA 2025)",
    month = jul,
    year = "2025",
    address = "Vienna, Austria",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/2025.bea-1.51/",
    doi = "10.18653/v1/2025.bea-1.51",
    pages = "708--715",
    ISBN = "979-8-89176-270-1",
    abstract = "This study investigates the use of ChatGPT for Automated Essay Scoring (AES) in assessing Italian middle school students' written texts. Using rubrics targeting grammar, coherence and argumentation, we compare AI-generated feedback with that of a human teacher on a newly collected corpus of students' essays. Despite some differences, ChatGPT provided detailed and timely feedback that complements the teacher{'}s role. These findings underscore the potential of generative AI to improve the assessment of writing, providing useful insights for educators and supporting students in developing their writing skills."
}

Download as File

<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="mirabella-brunato-2025-exploring">
    <titleInfo>
        <title>Exploring LLM-Based Assessment of Italian Middle School Writing: A Pilot Study</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Adriana</namePart>
        <namePart type="family">Mirabella</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Dominique</namePart>
        <namePart type="family">Brunato</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2025-07</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the 20th Workshop on Innovative Use of NLP for Building Educational Applications (BEA 2025)</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Ekaterina</namePart>
            <namePart type="family">Kochmar</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Bashar</namePart>
            <namePart type="family">Alhafni</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Marie</namePart>
            <namePart type="family">Bexte</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Jill</namePart>
            <namePart type="family">Burstein</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Andrea</namePart>
            <namePart type="family">Horbach</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Ronja</namePart>
            <namePart type="family">Laarmann-Quante</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Anaïs</namePart>
            <namePart type="family">Tack</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Victoria</namePart>
            <namePart type="family">Yaneva</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Zheng</namePart>
            <namePart type="family">Yuan</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Vienna, Austria</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
        <identifier type="isbn">979-8-89176-270-1</identifier>
    </relatedItem>
    <abstract>This study investigates the use of ChatGPT for Automated Essay Scoring (AES) in assessing Italian middle school students’ written texts. Using rubrics targeting grammar, coherence and argumentation, we compare AI-generated feedback with that of a human teacher on a newly collected corpus of students’ essays. Despite some differences, ChatGPT provided detailed and timely feedback that complements the teacher’s role. These findings underscore the potential of generative AI to improve the assessment of writing, providing useful insights for educators and supporting students in developing their writing skills.</abstract>
    <identifier type="citekey">mirabella-brunato-2025-exploring</identifier>
    <identifier type="doi">10.18653/v1/2025.bea-1.51</identifier>
    <location>
        <url>https://aclanthology.org/2025.bea-1.51/</url>
    </location>
    <part>
        <date>2025-07</date>
        <extent unit="page">
            <start>708</start>
            <end>715</end>
        </extent>
    </part>
</mods>
</modsCollection>

Download as File

%0 Conference Proceedings
%T Exploring LLM-Based Assessment of Italian Middle School Writing: A Pilot Study
%A Mirabella, Adriana
%A Brunato, Dominique
%Y Kochmar, Ekaterina
%Y Alhafni, Bashar
%Y Bexte, Marie
%Y Burstein, Jill
%Y Horbach, Andrea
%Y Laarmann-Quante, Ronja
%Y Tack, Anaïs
%Y Yaneva, Victoria
%Y Yuan, Zheng
%S Proceedings of the 20th Workshop on Innovative Use of NLP for Building Educational Applications (BEA 2025)
%D 2025
%8 July
%I Association for Computational Linguistics
%C Vienna, Austria
%@ 979-8-89176-270-1
%F mirabella-brunato-2025-exploring
%X This study investigates the use of ChatGPT for Automated Essay Scoring (AES) in assessing Italian middle school students’ written texts. Using rubrics targeting grammar, coherence and argumentation, we compare AI-generated feedback with that of a human teacher on a newly collected corpus of students’ essays. Despite some differences, ChatGPT provided detailed and timely feedback that complements the teacher’s role. These findings underscore the potential of generative AI to improve the assessment of writing, providing useful insights for educators and supporting students in developing their writing skills.
%R 10.18653/v1/2025.bea-1.51
%U https://aclanthology.org/2025.bea-1.51/
%U https://doi.org/10.18653/v1/2025.bea-1.51
%P 708-715

Download as File

Markdown (Informal)

[Exploring LLM-Based Assessment of Italian Middle School Writing: A Pilot Study](https://aclanthology.org/2025.bea-1.51/) (Mirabella & Brunato, BEA 2025)

Exploring LLM-Based Assessment of Italian Middle School Writing: A Pilot Study (Mirabella & Brunato, BEA 2025)

ACL

Adriana Mirabella and Dominique Brunato. 2025. Exploring LLM-Based Assessment of Italian Middle School Writing: A Pilot Study. In Proceedings of the 20th Workshop on Innovative Use of NLP for Building Educational Applications (BEA 2025), pages 708–715, Vienna, Austria. Association for Computational Linguistics.