WMT24 Test Suite: Gender Resolution in Speaker-Listener Dialogue Roles

Hillary Dawkins, Isar Nejadgholi, Chi-Kiu Lo


Abstract
We assess the difficulty of gender resolution in literary-style dialogue settings and the influence of gender stereotypes. Instances of the test suite contain spoken dialogue interleaved with external meta-context about the characters and the manner of speaking. We find that character and manner stereotypes outside of the dialogue significantly impact the gender agreement of referents within the dialogue.
Anthology ID:
2024.wmt-1.25
Volume:
Proceedings of the Ninth Conference on Machine Translation
Month:
November
Year:
2024
Address:
Miami, Florida, USA
Editors:
Barry Haddow, Tom Kocmi, Philipp Koehn, Christof Monz
Venue:
WMT
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
307–326
Language:
URL:
https://aclanthology.org/2024.wmt-1.25
DOI:
Bibkey:
Cite (ACL):
Hillary Dawkins, Isar Nejadgholi, and Chi-Kiu Lo. 2024. WMT24 Test Suite: Gender Resolution in Speaker-Listener Dialogue Roles. In Proceedings of the Ninth Conference on Machine Translation, pages 307–326, Miami, Florida, USA. Association for Computational Linguistics.
Cite (Informal):
WMT24 Test Suite: Gender Resolution in Speaker-Listener Dialogue Roles (Dawkins et al., WMT 2024)
Copy Citation:
PDF:
https://aclanthology.org/2024.wmt-1.25.pdf