<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="prange-chersoni-2023-empirical">
    <titleInfo>
        <title>Empirical Sufficiency Lower Bounds for Language Modeling with Locally-Bootstrapped Semantic Structures</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Jakob</namePart>
        <namePart type="family">Prange</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Emmanuele</namePart>
        <namePart type="family">Chersoni</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2023-07</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the 12th Joint Conference on Lexical and Computational Semantics (*SEM 2023)</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Alexis</namePart>
            <namePart type="family">Palmer</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Jose</namePart>
            <namePart type="family">Camacho-collados</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Toronto, Canada</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>In this work we build upon negative results from an attempt at language modeling with predicted semantic structure, in order to establish empirical lower bounds on what could have made the attempt successful. More specifically, we design a concise binary vector representation of semantic structure at the lexical level and evaluate in-depth how good an incremental tagger needs to be in order to achieve better-than-baseline performance with an end-to-end semantic-bootstrapping language model. We envision such a system as consisting of a (pretrained) sequential-neural component and a hierarchical-symbolic component working together to generate text with low surprisal and high linguistic interpretability. We find that (a) dimensionality of the semantic vector representation can be dramatically reduced without losing its main advantages and (b) lower bounds on prediction quality cannot be established via a single score alone, but need to take the distributions of signal and noise into account.</abstract>
    <identifier type="citekey">prange-chersoni-2023-empirical</identifier>
    <identifier type="doi">10.18653/v1/2023.starsem-1.40</identifier>
    <location>
        <url>https://aclanthology.org/2023.starsem-1.40</url>
    </location>
    <part>
        <date>2023-07</date>
        <extent unit="page">
            <start>456</start>
            <end>468</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Empirical Sufficiency Lower Bounds for Language Modeling with Locally-Bootstrapped Semantic Structures
%A Prange, Jakob
%A Chersoni, Emmanuele
%Y Palmer, Alexis
%Y Camacho-collados, Jose
%S Proceedings of the 12th Joint Conference on Lexical and Computational Semantics (*SEM 2023)
%D 2023
%8 July
%I Association for Computational Linguistics
%C Toronto, Canada
%F prange-chersoni-2023-empirical
%X In this work we build upon negative results from an attempt at language modeling with predicted semantic structure, in order to establish empirical lower bounds on what could have made the attempt successful. More specifically, we design a concise binary vector representation of semantic structure at the lexical level and evaluate in-depth how good an incremental tagger needs to be in order to achieve better-than-baseline performance with an end-to-end semantic-bootstrapping language model. We envision such a system as consisting of a (pretrained) sequential-neural component and a hierarchical-symbolic component working together to generate text with low surprisal and high linguistic interpretability. We find that (a) dimensionality of the semantic vector representation can be dramatically reduced without losing its main advantages and (b) lower bounds on prediction quality cannot be established via a single score alone, but need to take the distributions of signal and noise into account.
%R 10.18653/v1/2023.starsem-1.40
%U https://aclanthology.org/2023.starsem-1.40
%U https://doi.org/10.18653/v1/2023.starsem-1.40
%P 456-468

Markdown (Informal)

[Empirical Sufficiency Lower Bounds for Language Modeling with Locally-Bootstrapped Semantic Structures](https://aclanthology.org/2023.starsem-1.40) (Prange & Chersoni, *SEM 2023)

• Empirical Sufficiency Lower Bounds for Language Modeling with Locally-Bootstrapped Semantic Structures (Prange & Chersoni, *SEM 2023)

ACL

• Jakob Prange and Emmanuele Chersoni. 2023. Empirical Sufficiency Lower Bounds for Language Modeling with Locally-Bootstrapped Semantic Structures. In Proceedings of the 12th Joint Conference on Lexical and Computational Semantics (*SEM 2023), pages 456–468, Toronto, Canada. Association for Computational Linguistics.