Codec at SemEval-2022 Task 5: Multi-Modal Multi-Transformer Misogynous Meme Classification Framework

Ahmed Mahran, Carlo Alessandro Borella, Konstantinos Perifanos


Abstract
In this paper we describe our work towards building a generic framework for both multi-modal embedding and multi-label binary classification tasks, while participating in task 5 (Multimedia Automatic Misogyny Identification) of SemEval 2022 competition. Since pretraining deep models from scratch is a resource and data hungry task, our approach is based on three main strategies. We combine different state-of-the-art architectures to capture a wide spectrum of semantic signals from the multi-modal input. We employ a multi-task learning scheme to be able to use multiple datasets from the same knowledge domain to help increase the model’s performance. We also use multiple objectives to regularize and fine tune different system components.
Anthology ID:
2022.semeval-1.93
Volume:
Proceedings of the 16th International Workshop on Semantic Evaluation (SemEval-2022)
Month:
July
Year:
2022
Address:
Seattle, United States
Editors:
Guy Emerson, Natalie Schluter, Gabriel Stanovsky, Ritesh Kumar, Alexis Palmer, Nathan Schneider, Siddharth Singh, Shyam Ratan
Venue:
SemEval
SIG:
SIGLEX
Publisher:
Association for Computational Linguistics
Note:
Pages:
679–688
Language:
URL:
https://aclanthology.org/2022.semeval-1.93
DOI:
10.18653/v1/2022.semeval-1.93
Bibkey:
Cite (ACL):
Ahmed Mahran, Carlo Alessandro Borella, and Konstantinos Perifanos. 2022. Codec at SemEval-2022 Task 5: Multi-Modal Multi-Transformer Misogynous Meme Classification Framework. In Proceedings of the 16th International Workshop on Semantic Evaluation (SemEval-2022), pages 679–688, Seattle, United States. Association for Computational Linguistics.
Cite (Informal):
Codec at SemEval-2022 Task 5: Multi-Modal Multi-Transformer Misogynous Meme Classification Framework (Mahran et al., SemEval 2022)
Copy Citation:
PDF:
https://aclanthology.org/2022.semeval-1.93.pdf
Code
 ahmed-mahran/mami2022
Data
Hateful Memes