<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="ma-2024-evaluating">
    <titleInfo>
        <title>Evaluating Lexical Aspect with Large Language Models</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Bolei</namePart>
        <namePart type="family">Ma</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2024-08</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the Workshop on Cognitive Modeling and Computational Linguistics</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Tatsuki</namePart>
            <namePart type="family">Kuribayashi</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Giulia</namePart>
            <namePart type="family">Rambelli</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Ece</namePart>
            <namePart type="family">Takmaz</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Philipp</namePart>
            <namePart type="family">Wicke</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Yohei</namePart>
            <namePart type="family">Oseki</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Bangkok, Thailand</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>In this study, we explore the proficiency of large language models (LLMs) in understanding two key lexical aspects: duration (durative/stative) and telicity (telic/atelic). Through experiments on datasets featuring sentences, verbs, and verb positions, we prompt the LLMs to identify aspectual features of verbs in sentences. Our findings reveal that certain LLMs, particularly those closed-source ones, are able to capture information on duration and telicity, albeit with some performance variations and weaker results compared to the baseline. By employing prompts at three levels (sentence-only, sentence with verb, and sentence with verb and its position), we demonstrate that integrating verb information generally enhances performance in aspectual feature recognition, though it introduces instability. We call for future research to look deeper into methods aimed at optimizing LLMs for aspectual feature comprehension.</abstract>
    <identifier type="citekey">ma-2024-evaluating</identifier>
    <identifier type="doi">10.18653/v1/2024.cmcl-1.11</identifier>
    <location>
        <url>https://aclanthology.org/2024.cmcl-1.11</url>
    </location>
    <part>
        <date>2024-08</date>
        <extent unit="page">
            <start>123</start>
            <end>131</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Evaluating Lexical Aspect with Large Language Models
%A Ma, Bolei
%Y Kuribayashi, Tatsuki
%Y Rambelli, Giulia
%Y Takmaz, Ece
%Y Wicke, Philipp
%Y Oseki, Yohei
%S Proceedings of the Workshop on Cognitive Modeling and Computational Linguistics
%D 2024
%8 August
%I Association for Computational Linguistics
%C Bangkok, Thailand
%F ma-2024-evaluating
%X In this study, we explore the proficiency of large language models (LLMs) in understanding two key lexical aspects: duration (durative/stative) and telicity (telic/atelic). Through experiments on datasets featuring sentences, verbs, and verb positions, we prompt the LLMs to identify aspectual features of verbs in sentences. Our findings reveal that certain LLMs, particularly those closed-source ones, are able to capture information on duration and telicity, albeit with some performance variations and weaker results compared to the baseline. By employing prompts at three levels (sentence-only, sentence with verb, and sentence with verb and its position), we demonstrate that integrating verb information generally enhances performance in aspectual feature recognition, though it introduces instability. We call for future research to look deeper into methods aimed at optimizing LLMs for aspectual feature comprehension.
%R 10.18653/v1/2024.cmcl-1.11
%U https://aclanthology.org/2024.cmcl-1.11
%U https://doi.org/10.18653/v1/2024.cmcl-1.11
%P 123-131

Markdown (Informal)

[Evaluating Lexical Aspect with Large Language Models](https://aclanthology.org/2024.cmcl-1.11) (Ma, CMCL-WS 2024)

• Evaluating Lexical Aspect with Large Language Models (Ma, CMCL-WS 2024)

ACL

• Bolei Ma. 2024. Evaluating Lexical Aspect with Large Language Models. In Proceedings of the Workshop on Cognitive Modeling and Computational Linguistics, pages 123–131, Bangkok, Thailand. Association for Computational Linguistics.