<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="sant-etal-2024-power">
    <titleInfo>
        <title>The power of Prompts: Evaluating and Mitigating Gender Bias in MT with LLMs</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Aleix</namePart>
        <namePart type="family">Sant</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Carlos</namePart>
        <namePart type="family">Escolano</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Audrey</namePart>
        <namePart type="family">Mash</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Francesca</namePart>
        <namePart type="family">De Luca Fornaciari</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Maite</namePart>
        <namePart type="family">Melero</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2024-08</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the 5th Workshop on Gender Bias in Natural Language Processing (GeBNLP)</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Agnieszka</namePart>
            <namePart type="family">Faleńska</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Christine</namePart>
            <namePart type="family">Basta</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Marta</namePart>
            <namePart type="family">Costa-jussà</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Seraphina</namePart>
            <namePart type="family">Goldfarb-Tarrant</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Debora</namePart>
            <namePart type="family">Nozza</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Bangkok, Thailand</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>This paper studies gender bias in machine translation through the lens of Large Language Models (LLMs). Four widely-used test sets are employed to benchmark various base LLMs, comparing their translation quality and gender bias against state-of-the-art Neural Machine Translation (NMT) models for English to Catalan (En → Ca) and English to Spanish (En → Es) translation directions. Our findings reveal pervasive gender bias across all models, with base LLMs exhibiting a higher degree of bias compared to NMT models.To combat this bias, we explore prompting engineering techniques applied to an instruction-tuned LLM. We identify a prompt structure that significantly reduces gender bias by up to 12% on the WinoMT evaluation dataset compared to more straightforward prompts. These results significantly reduce the gender bias accuracy gap between LLMs and traditional NMT systems.</abstract>
    <identifier type="citekey">sant-etal-2024-power</identifier>
    <identifier type="doi">10.18653/v1/2024.gebnlp-1.7</identifier>
    <location>
        <url>https://aclanthology.org/2024.gebnlp-1.7</url>
    </location>
    <part>
        <date>2024-08</date>
        <extent unit="page">
            <start>94</start>
            <end>139</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T The power of Prompts: Evaluating and Mitigating Gender Bias in MT with LLMs
%A Sant, Aleix
%A Escolano, Carlos
%A Mash, Audrey
%A De Luca Fornaciari, Francesca
%A Melero, Maite
%Y Faleńska, Agnieszka
%Y Basta, Christine
%Y Costa-jussà, Marta
%Y Goldfarb-Tarrant, Seraphina
%Y Nozza, Debora
%S Proceedings of the 5th Workshop on Gender Bias in Natural Language Processing (GeBNLP)
%D 2024
%8 August
%I Association for Computational Linguistics
%C Bangkok, Thailand
%F sant-etal-2024-power
%X This paper studies gender bias in machine translation through the lens of Large Language Models (LLMs). Four widely-used test sets are employed to benchmark various base LLMs, comparing their translation quality and gender bias against state-of-the-art Neural Machine Translation (NMT) models for English to Catalan (En → Ca) and English to Spanish (En → Es) translation directions. Our findings reveal pervasive gender bias across all models, with base LLMs exhibiting a higher degree of bias compared to NMT models.To combat this bias, we explore prompting engineering techniques applied to an instruction-tuned LLM. We identify a prompt structure that significantly reduces gender bias by up to 12% on the WinoMT evaluation dataset compared to more straightforward prompts. These results significantly reduce the gender bias accuracy gap between LLMs and traditional NMT systems.
%R 10.18653/v1/2024.gebnlp-1.7
%U https://aclanthology.org/2024.gebnlp-1.7
%U https://doi.org/10.18653/v1/2024.gebnlp-1.7
%P 94-139

Markdown (Informal)

[The power of Prompts: Evaluating and Mitigating Gender Bias in MT with LLMs](https://aclanthology.org/2024.gebnlp-1.7) (Sant et al., GeBNLP-WS 2024)

• The power of Prompts: Evaluating and Mitigating Gender Bias in MT with LLMs (Sant et al., GeBNLP-WS 2024)

ACL

• Aleix Sant, Carlos Escolano, Audrey Mash, Francesca De Luca Fornaciari, and Maite Melero. 2024. The power of Prompts: Evaluating and Mitigating Gender Bias in MT with LLMs. In Proceedings of the 5th Workshop on Gender Bias in Natural Language Processing (GeBNLP), pages 94–139, Bangkok, Thailand. Association for Computational Linguistics.