<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="lan-etal-2023-benchmarking">
    <titleInfo>
        <title>Benchmarking Neural Network Generalization for Grammar Induction</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Nur</namePart>
        <namePart type="family">Lan</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Emmanuel</namePart>
        <namePart type="family">Chemla</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Roni</namePart>
        <namePart type="family">Katzir</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2023-09</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the 2023 CLASP Conference on Learning with Small Data (LSD)</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Ellen</namePart>
            <namePart type="family">Breitholtz</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Shalom</namePart>
            <namePart type="family">Lappin</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Sharid</namePart>
            <namePart type="family">Loaiciga</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Nikolai</namePart>
            <namePart type="family">Ilinykh</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Simon</namePart>
            <namePart type="family">Dobnik</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Gothenburg, Sweden</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>How well do neural networks generalize? Even for grammar induction tasks, where the target generalization is fully known, previous works have left the question open, testing very limited ranges beyond the training set and using different success criteria. We provide a measure of neural network generalization based on fully specified formal languages. Given a model and a formal grammar, the method assigns a generalization score representing how well a model generalizes to unseen samples in inverse relation to the amount of data it was trained on. The benchmark includes languages such as aⁿbⁿ, aⁿbⁿcⁿ, aⁿb^mcⁿ+m, and Dyck-1 and 2. We evaluate selected architectures using the benchmark and find that networks trained with a Minimum Description Length objective (MDL) generalize better and using less data than networks trained using standard loss functions. The benchmark is available at https://github.com/taucompling/bliss.</abstract>
    <identifier type="citekey">lan-etal-2023-benchmarking</identifier>
    <location>
        <url>https://aclanthology.org/2023.clasp-1.15</url>
    </location>
    <part>
        <date>2023-09</date>
        <extent unit="page">
            <start>131</start>
            <end>140</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Benchmarking Neural Network Generalization for Grammar Induction
%A Lan, Nur
%A Chemla, Emmanuel
%A Katzir, Roni
%Y Breitholtz, Ellen
%Y Lappin, Shalom
%Y Loaiciga, Sharid
%Y Ilinykh, Nikolai
%Y Dobnik, Simon
%S Proceedings of the 2023 CLASP Conference on Learning with Small Data (LSD)
%D 2023
%8 September
%I Association for Computational Linguistics
%C Gothenburg, Sweden
%F lan-etal-2023-benchmarking
%X How well do neural networks generalize? Even for grammar induction tasks, where the target generalization is fully known, previous works have left the question open, testing very limited ranges beyond the training set and using different success criteria. We provide a measure of neural network generalization based on fully specified formal languages. Given a model and a formal grammar, the method assigns a generalization score representing how well a model generalizes to unseen samples in inverse relation to the amount of data it was trained on. The benchmark includes languages such as aⁿbⁿ, aⁿbⁿcⁿ, aⁿb^mcⁿ+m, and Dyck-1 and 2. We evaluate selected architectures using the benchmark and find that networks trained with a Minimum Description Length objective (MDL) generalize better and using less data than networks trained using standard loss functions. The benchmark is available at https://github.com/taucompling/bliss.
%U https://aclanthology.org/2023.clasp-1.15
%P 131-140

Markdown (Informal)

[Benchmarking Neural Network Generalization for Grammar Induction](https://aclanthology.org/2023.clasp-1.15) (Lan et al., CLASP 2023)

• Benchmarking Neural Network Generalization for Grammar Induction (Lan et al., CLASP 2023)

ACL

• Nur Lan, Emmanuel Chemla, and Roni Katzir. 2023. Benchmarking Neural Network Generalization for Grammar Induction. In Proceedings of the 2023 CLASP Conference on Learning with Small Data (LSD), pages 131–140, Gothenburg, Sweden. Association for Computational Linguistics.