<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="li-etal-2025-assessing-llms">
    <titleInfo>
        <title>Assessing LLMs’ Understanding of Structural Contrasts in the Lexicon</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Shuxu</namePart>
        <namePart type="family">Li</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Antoine</namePart>
        <namePart type="family">Venant</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Philippe</namePart>
        <namePart type="family">Langlais</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">François</namePart>
        <namePart type="family">Lareau</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2025-09</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the 16th International Conference on Computational Semantics</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Kilian</namePart>
            <namePart type="family">Evang</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Laura</namePart>
            <namePart type="family">Kallmeyer</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Sylvain</namePart>
            <namePart type="family">Pogodalla</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Düsseldorf, Germany</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
        <identifier type="isbn">979-8-89176-316-6</identifier>
    </relatedItem>
    <abstract>We present a new benchmark to evaluate the lexical competence of large language models (LLMs), built on a hierarchical classification of lexical functions (LFs) within the Meaning-Text Theory (MTT) framework. Based on a dataset called French Lexical Network (LN-fr), the benchmark employs contrastive tasks to probe the models’ sensitivity to fine-grained paradigmatic and syntagmatic distinctions. Our results show that performance varies significantly across different LFs and systematically declines with increased distinction granularity, highlighting current LLMs’ limitations in relational and structured lexical understanding.</abstract>
    <identifier type="citekey">li-etal-2025-assessing-llms</identifier>
    <location>
        <url>https://aclanthology.org/2025.iwcs-main.9/</url>
    </location>
    <part>
        <date>2025-09</date>
        <extent unit="page">
            <start>98</start>
            <end>109</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Assessing LLMs’ Understanding of Structural Contrasts in the Lexicon
%A Li, Shuxu
%A Venant, Antoine
%A Langlais, Philippe
%A Lareau, François
%Y Evang, Kilian
%Y Kallmeyer, Laura
%Y Pogodalla, Sylvain
%S Proceedings of the 16th International Conference on Computational Semantics
%D 2025
%8 September
%I Association for Computational Linguistics
%C Düsseldorf, Germany
%@ 979-8-89176-316-6
%F li-etal-2025-assessing-llms
%X We present a new benchmark to evaluate the lexical competence of large language models (LLMs), built on a hierarchical classification of lexical functions (LFs) within the Meaning-Text Theory (MTT) framework. Based on a dataset called French Lexical Network (LN-fr), the benchmark employs contrastive tasks to probe the models’ sensitivity to fine-grained paradigmatic and syntagmatic distinctions. Our results show that performance varies significantly across different LFs and systematically declines with increased distinction granularity, highlighting current LLMs’ limitations in relational and structured lexical understanding.
%U https://aclanthology.org/2025.iwcs-main.9/
%P 98-109

Markdown (Informal)

[Assessing LLMs’ Understanding of Structural Contrasts in the Lexicon](https://aclanthology.org/2025.iwcs-main.9/) (Li et al., IWCS 2025)

• Assessing LLMs’ Understanding of Structural Contrasts in the Lexicon (Li et al., IWCS 2025)

ACL

• Shuxu Li, Antoine Venant, Philippe Langlais, and François Lareau. 2025. Assessing LLMs’ Understanding of Structural Contrasts in the Lexicon. In Proceedings of the 16th International Conference on Computational Semantics, pages 98–109, Düsseldorf, Germany. Association for Computational Linguistics.