ReSeTOX: Re-learning attention weights for toxicity mitigation in machine translation

Javier García Gilabert, Carlos Escolano, Marta Costa-jussà


Abstract
Our proposed method, RESETOX (REdoSEarch if TOXic), addresses the issue ofNeural Machine Translation (NMT) gener-ating translation outputs that contain toxicwords not present in the input. The ob-jective is to mitigate the introduction oftoxic language without the need for re-training. In the case of identified addedtoxicity during the inference process, RE-SETOX dynamically adjusts the key-valueself-attention weights and re-evaluates thebeam search hypotheses. Experimental re-sults demonstrate that RESETOX achievesa remarkable 57% reduction in added tox-icity while maintaining an average trans-lation quality of 99.5% across 164 lan-guages. Our code is available at: https://github.com
Anthology ID:
2024.eamt-1.8
Volume:
Proceedings of the 25th Annual Conference of the European Association for Machine Translation (Volume 1)
Month:
June
Year:
2024
Address:
Sheffield, UK
Editors:
Carolina Scarton, Charlotte Prescott, Chris Bayliss, Chris Oakley, Joanna Wright, Stuart Wrigley, Xingyi Song, Edward Gow-Smith, Rachel Bawden, Víctor M Sánchez-Cartagena, Patrick Cadwell, Ekaterina Lapshinova-Koltunski, Vera Cabarrão, Konstantinos Chatzitheodorou, Mary Nurminen, Diptesh Kanojia, Helena Moniz
Venue:
EAMT
SIG:
Publisher:
European Association for Machine Translation (EAMT)
Note:
Pages:
37–58
Language:
URL:
https://aclanthology.org/2024.eamt-1.8
DOI:
Bibkey:
Cite (ACL):
Javier García Gilabert, Carlos Escolano, and Marta Costa-jussà. 2024. ReSeTOX: Re-learning attention weights for toxicity mitigation in machine translation. In Proceedings of the 25th Annual Conference of the European Association for Machine Translation (Volume 1), pages 37–58, Sheffield, UK. European Association for Machine Translation (EAMT).
Cite (Informal):
ReSeTOX: Re-learning attention weights for toxicity mitigation in machine translation (García Gilabert et al., EAMT 2024)
Copy Citation:
PDF:
https://aclanthology.org/2024.eamt-1.8.pdf