<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="purushothama-etal-2024-getting">
    <titleInfo>
        <title>Getting The Most Out of Your Training Data: Exploring Unsupervised Tasks for Morphological Inflection</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Abhishek</namePart>
        <namePart type="family">Purushothama</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Adam</namePart>
        <namePart type="family">Wiemerslage</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Katharina</namePart>
        <namePart type="family">Wense</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2024-11</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Yaser</namePart>
            <namePart type="family">Al-Onaizan</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Mohit</namePart>
            <namePart type="family">Bansal</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Yun-Nung</namePart>
            <namePart type="family">Chen</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Miami, Florida, USA</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>Pre-trained transformers such as BERT have been shown to be effective in many natural language tasks. However, they are under-explored for character-level sequence to sequence tasks. In this work, we investigate pre-training transformers for the character-level task of morphological inflection in several languages. We compare various training setups and secondary tasks where unsupervised data taken directly from the target task is used. We show that training on secondary unsupervised tasks increases inflection performance even without any external data, suggesting that models learn from additional unsupervised tasks themselves—not just from additional data. We also find that this does not hold true for specific combinations of secondary task and training setup, which has interesting implications for denoising objectives in character-level tasks.</abstract>
    <identifier type="citekey">purushothama-etal-2024-getting</identifier>
    <location>
        <url>https://aclanthology.org/2024.emnlp-main.1055</url>
    </location>
    <part>
        <date>2024-11</date>
        <extent unit="page">
            <start>18955</start>
            <end>18970</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Getting The Most Out of Your Training Data: Exploring Unsupervised Tasks for Morphological Inflection
%A Purushothama, Abhishek
%A Wiemerslage, Adam
%A Wense, Katharina
%Y Al-Onaizan, Yaser
%Y Bansal, Mohit
%Y Chen, Yun-Nung
%S Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing
%D 2024
%8 November
%I Association for Computational Linguistics
%C Miami, Florida, USA
%F purushothama-etal-2024-getting
%X Pre-trained transformers such as BERT have been shown to be effective in many natural language tasks. However, they are under-explored for character-level sequence to sequence tasks. In this work, we investigate pre-training transformers for the character-level task of morphological inflection in several languages. We compare various training setups and secondary tasks where unsupervised data taken directly from the target task is used. We show that training on secondary unsupervised tasks increases inflection performance even without any external data, suggesting that models learn from additional unsupervised tasks themselves—not just from additional data. We also find that this does not hold true for specific combinations of secondary task and training setup, which has interesting implications for denoising objectives in character-level tasks.
%U https://aclanthology.org/2024.emnlp-main.1055
%P 18955-18970

Markdown (Informal)

[Getting The Most Out of Your Training Data: Exploring Unsupervised Tasks for Morphological Inflection](https://aclanthology.org/2024.emnlp-main.1055) (Purushothama et al., EMNLP 2024)

• Getting The Most Out of Your Training Data: Exploring Unsupervised Tasks for Morphological Inflection (Purushothama et al., EMNLP 2024)

ACL

• Abhishek Purushothama, Adam Wiemerslage, and Katharina Wense. 2024. Getting The Most Out of Your Training Data: Exploring Unsupervised Tasks for Morphological Inflection. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 18955–18970, Miami, Florida, USA. Association for Computational Linguistics.