<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="liu-prudhommeaux-2022-data">
    <titleInfo>
        <title>Data-driven Model Generalizability in Crosslinguistic Low-resource Morphological Segmentation</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Zoey</namePart>
        <namePart type="family">Liu</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Emily</namePart>
        <namePart type="family">Prud’hommeaux</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2022</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <genre authority="bibutilsgt">journal article</genre>
    <relatedItem type="host">
        <titleInfo>
            <title>Transactions of the Association for Computational Linguistics</title>
        </titleInfo>
        <originInfo>
            <issuance>continuing</issuance>
            <publisher>MIT Press</publisher>
            <place>
                <placeTerm type="text">Cambridge, MA</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">periodical</genre>
        <genre authority="bibutilsgt">academic journal</genre>
    </relatedItem>
    <abstract>Common designs of model evaluation typically focus on monolingual settings, where different models are compared according to their performance on a single data set that is assumed to be representative of all possible data for the task at hand. While this may be reasonable for a large data set, this assumption is difficult to maintain in low-resource scenarios, where artifacts of the data collection can yield data sets that are outliers, potentially making conclusions about model performance coincidental. To address these concerns, we investigate model generalizability in crosslinguistic low-resource scenarios. Using morphological segmentation as the test case, we compare three broad classes of models with different parameterizations, taking data from 11 languages across 6 language families. In each experimental setting, we evaluate all models on a first data set, then examine their performance consistency when introducing new randomly sampled data sets with the same size and when applying the trained models to unseen test sets of varying sizes. The results demonstrate that the extent of model generalization depends on the characteristics of the data set, and does not necessarily rely heavily on the data set size. Among the characteristics that we studied, the ratio of morpheme overlap and that of the average number of morphemes per word between the training and test sets are the two most prominent factors. Our findings suggest that future work should adopt random sampling to construct data sets with different sizes in order to make more responsible claims about model evaluation.</abstract>
    <identifier type="citekey">liu-prudhommeaux-2022-data</identifier>
    <identifier type="doi">10.1162/tacl_a_00467</identifier>
    <location>
        <url>https://aclanthology.org/2022.tacl-1.23</url>
    </location>
    <part>
        <date>2022</date>
        <detail type="volume"><number>10</number></detail>
        <extent unit="page">
            <start>393</start>
            <end>413</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Journal Article
%T Data-driven Model Generalizability in Crosslinguistic Low-resource Morphological Segmentation
%A Liu, Zoey
%A Prud’hommeaux, Emily
%J Transactions of the Association for Computational Linguistics
%D 2022
%V 10
%I MIT Press
%C Cambridge, MA
%F liu-prudhommeaux-2022-data
%X Common designs of model evaluation typically focus on monolingual settings, where different models are compared according to their performance on a single data set that is assumed to be representative of all possible data for the task at hand. While this may be reasonable for a large data set, this assumption is difficult to maintain in low-resource scenarios, where artifacts of the data collection can yield data sets that are outliers, potentially making conclusions about model performance coincidental. To address these concerns, we investigate model generalizability in crosslinguistic low-resource scenarios. Using morphological segmentation as the test case, we compare three broad classes of models with different parameterizations, taking data from 11 languages across 6 language families. In each experimental setting, we evaluate all models on a first data set, then examine their performance consistency when introducing new randomly sampled data sets with the same size and when applying the trained models to unseen test sets of varying sizes. The results demonstrate that the extent of model generalization depends on the characteristics of the data set, and does not necessarily rely heavily on the data set size. Among the characteristics that we studied, the ratio of morpheme overlap and that of the average number of morphemes per word between the training and test sets are the two most prominent factors. Our findings suggest that future work should adopt random sampling to construct data sets with different sizes in order to make more responsible claims about model evaluation.
%R 10.1162/tacl_a_00467
%U https://aclanthology.org/2022.tacl-1.23
%U https://doi.org/10.1162/tacl_a_00467
%P 393-413

Markdown (Informal)

[Data-driven Model Generalizability in Crosslinguistic Low-resource Morphological Segmentation](https://aclanthology.org/2022.tacl-1.23) (Liu & Prud’hommeaux, TACL 2022)

• Data-driven Model Generalizability in Crosslinguistic Low-resource Morphological Segmentation (Liu & Prud’hommeaux, TACL 2022)

ACL

• Zoey Liu and Emily Prud’hommeaux. 2022. Data-driven Model Generalizability in Crosslinguistic Low-resource Morphological Segmentation. Transactions of the Association for Computational Linguistics, 10:393–413.