<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="escolano-etal-2024-residual">
    <titleInfo>
        <title>Residual Dropout: A Simple Approach to Improve Transformer’s Data Efficiency</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Carlos</namePart>
        <namePart type="family">Escolano</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Francesca</namePart>
        <namePart type="family">De Luca Fornaciari</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Maite</namePart>
        <namePart type="family">Melero</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2024-05</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the 3rd Annual Meeting of the Special Interest Group on Under-resourced Languages @ LREC-COLING 2024</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Maite</namePart>
            <namePart type="family">Melero</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Sakriani</namePart>
            <namePart type="family">Sakti</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Claudia</namePart>
            <namePart type="family">Soria</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>ELRA and ICCL</publisher>
            <place>
                <placeTerm type="text">Torino, Italia</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>Transformer models often demand a vast amount of training data to achieve the desired level of performance. However, this data requirement poses a major challenge for low-resource languages seeking access to high-quality systems, particularly in tasks like Machine Translation. To address this issue, we propose adding Dropout to Transformer’s Residual Connections. Our experimental results demonstrate that this modification effectively mitigates overfitting during training, resulting in substantial performance gains of over 4 BLEU points on a dataset consisting of merely 10 thousand examples.</abstract>
    <identifier type="citekey">escolano-etal-2024-residual</identifier>
    <location>
        <url>https://aclanthology.org/2024.sigul-1.35</url>
    </location>
    <part>
        <date>2024-05</date>
        <extent unit="page">
            <start>294</start>
            <end>299</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Residual Dropout: A Simple Approach to Improve Transformer’s Data Efficiency
%A Escolano, Carlos
%A De Luca Fornaciari, Francesca
%A Melero, Maite
%Y Melero, Maite
%Y Sakti, Sakriani
%Y Soria, Claudia
%S Proceedings of the 3rd Annual Meeting of the Special Interest Group on Under-resourced Languages @ LREC-COLING 2024
%D 2024
%8 May
%I ELRA and ICCL
%C Torino, Italia
%F escolano-etal-2024-residual
%X Transformer models often demand a vast amount of training data to achieve the desired level of performance. However, this data requirement poses a major challenge for low-resource languages seeking access to high-quality systems, particularly in tasks like Machine Translation. To address this issue, we propose adding Dropout to Transformer’s Residual Connections. Our experimental results demonstrate that this modification effectively mitigates overfitting during training, resulting in substantial performance gains of over 4 BLEU points on a dataset consisting of merely 10 thousand examples.
%U https://aclanthology.org/2024.sigul-1.35
%P 294-299

Markdown (Informal)

[Residual Dropout: A Simple Approach to Improve Transformer’s Data Efficiency](https://aclanthology.org/2024.sigul-1.35) (Escolano et al., SIGUL-WS 2024)

• Residual Dropout: A Simple Approach to Improve Transformer’s Data Efficiency (Escolano et al., SIGUL-WS 2024)

ACL

• Carlos Escolano, Francesca De Luca Fornaciari, and Maite Melero. 2024. Residual Dropout: A Simple Approach to Improve Transformer’s Data Efficiency. In Proceedings of the 3rd Annual Meeting of the Special Interest Group on Under-resourced Languages @ LREC-COLING 2024, pages 294–299, Torino, Italia. ELRA and ICCL.