<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="savoldi-etal-2024-fbk">
    <titleInfo>
        <title>FBK@IWSLT Test Suites Task: Gender Bias evaluation with MuST-SHE</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Beatrice</namePart>
        <namePart type="family">Savoldi</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Marco</namePart>
        <namePart type="family">Gaido</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Matteo</namePart>
        <namePart type="family">Negri</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <name type="personal">
        <namePart type="given">Luisa</namePart>
        <namePart type="family">Bentivogli</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2024-08</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the 21st International Conference on Spoken Language Translation (IWSLT 2024)</title>
        </titleInfo>
        <name type="personal">
            <namePart type="given">Elizabeth</namePart>
            <namePart type="family">Salesky</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Marcello</namePart>
            <namePart type="family">Federico</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <name type="personal">
            <namePart type="given">Marine</namePart>
            <namePart type="family">Carpuat</namePart>
            <role>
                <roleTerm authority="marcrelator" type="text">editor</roleTerm>
            </role>
        </name>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Bangkok, Thailand (in-person and online)</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>This paper presents the FBK contribution to the IWSLT-2024 ‘Test suites’ shared subtask, part of the Offline Speech Translation Task. Our contribution consists of the MuST-SHE-IWSLT24 benchmark evaluation, designed to assess gender bias in speech translation. By focusing on the en-de language pair, we rely on a newly created test suite to investigate systems’ ability to correctly translate feminine and masculine gender. Our results indicate that – under realistic conditions – current ST systems achieve reasonable and comparable performance in correctly translating both feminine and masculine forms when contextual gender information is available. For ambiguous references to the speaker, however, we attest a consistent preference towards masculine gender, thus calling for future endeavours on the topic. Towards this goal we make MuST-SHE-IWSLT24 freely available at: https://mt.fbk.eu/must-she/</abstract>
    <identifier type="citekey">savoldi-etal-2024-fbk</identifier>
    <location>
        <url>https://aclanthology.org/2024.iwslt-1.10</url>
    </location>
    <part>
        <date>2024-08</date>
        <extent unit="page">
            <start>65</start>
            <end>71</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T FBK@IWSLT Test Suites Task: Gender Bias evaluation with MuST-SHE
%A Savoldi, Beatrice
%A Gaido, Marco
%A Negri, Matteo
%A Bentivogli, Luisa
%Y Salesky, Elizabeth
%Y Federico, Marcello
%Y Carpuat, Marine
%S Proceedings of the 21st International Conference on Spoken Language Translation (IWSLT 2024)
%D 2024
%8 August
%I Association for Computational Linguistics
%C Bangkok, Thailand (in-person and online)
%F savoldi-etal-2024-fbk
%X This paper presents the FBK contribution to the IWSLT-2024 ‘Test suites’ shared subtask, part of the Offline Speech Translation Task. Our contribution consists of the MuST-SHE-IWSLT24 benchmark evaluation, designed to assess gender bias in speech translation. By focusing on the en-de language pair, we rely on a newly created test suite to investigate systems’ ability to correctly translate feminine and masculine gender. Our results indicate that – under realistic conditions – current ST systems achieve reasonable and comparable performance in correctly translating both feminine and masculine forms when contextual gender information is available. For ambiguous references to the speaker, however, we attest a consistent preference towards masculine gender, thus calling for future endeavours on the topic. Towards this goal we make MuST-SHE-IWSLT24 freely available at: https://mt.fbk.eu/must-she/
%U https://aclanthology.org/2024.iwslt-1.10
%P 65-71

Markdown (Informal)

[FBK@IWSLT Test Suites Task: Gender Bias evaluation with MuST-SHE](https://aclanthology.org/2024.iwslt-1.10) (Savoldi et al., IWSLT 2024)

• FBK@IWSLT Test Suites Task: Gender Bias evaluation with MuST-SHE (Savoldi et al., IWSLT 2024)

ACL

• Beatrice Savoldi, Marco Gaido, Matteo Negri, and Luisa Bentivogli. 2024. FBK@IWSLT Test Suites Task: Gender Bias evaluation with MuST-SHE. In Proceedings of the 21st International Conference on Spoken Language Translation (IWSLT 2024), pages 65–71, Bangkok, Thailand (in-person and online). Association for Computational Linguistics.