<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="bogin-etal-2022-unobserved">
    <titleInfo>
        <title>Unobserved Local Structures Make Compositional Generalization Hard</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Ben</namePart>
        <namePart type="family">Bogin</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Shivanshu</namePart>
        <namePart type="family">Gupta</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Jonathan</namePart>
        <namePart type="family">Berant</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2022-12</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing</title>
        </titleInfo>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Abu Dhabi, United Arab Emirates</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>While recent work has shown that sequence-to-sequence models struggle to generalize to new compositions (termed compositional generalization), little is known on what makes compositional generalization hard on a particular test instance. In this work, we investigate the factors that make generalization to certain test instances challenging. We first substantiate that some examples are more difficult than others by showing that different models consistently fail or succeed on the same test instances. Then, we propose a criterion for the difficulty of an example: a test instance is hard if it contains a local structure that was not observed at training time. We formulate a simple decision rule based on this criterion and empirically show it predicts instance-level generalization well across 5 different semantic parsing datasets, substantially better than alternative decision rules. Last, we show local structures can be leveraged for creating difficult adversarial compositional splits and also to improve compositional generalization under limited training budgets by strategically selecting examples for the training set.</abstract>
    <identifier type="citekey">bogin-etal-2022-unobserved</identifier>
    <location>
        <url>https://aclanthology.org/2022.emnlp-main.175</url>
    </location>
    <part>
        <date>2022-12</date>
        <extent unit="page">
            <start>2731</start>
            <end>2747</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Unobserved Local Structures Make Compositional Generalization Hard
%A Bogin, Ben
%A Gupta, Shivanshu
%A Berant, Jonathan
%S Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing
%D 2022
%8 December
%I Association for Computational Linguistics
%C Abu Dhabi, United Arab Emirates
%F bogin-etal-2022-unobserved
%X While recent work has shown that sequence-to-sequence models struggle to generalize to new compositions (termed compositional generalization), little is known on what makes compositional generalization hard on a particular test instance. In this work, we investigate the factors that make generalization to certain test instances challenging. We first substantiate that some examples are more difficult than others by showing that different models consistently fail or succeed on the same test instances. Then, we propose a criterion for the difficulty of an example: a test instance is hard if it contains a local structure that was not observed at training time. We formulate a simple decision rule based on this criterion and empirically show it predicts instance-level generalization well across 5 different semantic parsing datasets, substantially better than alternative decision rules. Last, we show local structures can be leveraged for creating difficult adversarial compositional splits and also to improve compositional generalization under limited training budgets by strategically selecting examples for the training set.
%U https://aclanthology.org/2022.emnlp-main.175
%P 2731-2747

Markdown (Informal)

[Unobserved Local Structures Make Compositional Generalization Hard](https://aclanthology.org/2022.emnlp-main.175) (Bogin et al., EMNLP 2022)

• Unobserved Local Structures Make Compositional Generalization Hard (Bogin et al., EMNLP 2022)

ACL

• Ben Bogin, Shivanshu Gupta, and Jonathan Berant. 2022. Unobserved Local Structures Make Compositional Generalization Hard. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 2731–2747, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics.