TIB-VA at SemEval-2022 Task 5: A Multimodal Architecture for the Detection and Classification of Misogynous Memes

Sherzod Hakimov, Gullal Singh Cheema, Ralph Ewerth


Abstract
The detection of offensive, hateful content on social media is a challenging problem that affects many online users on a daily basis. Hateful content is often used to target a group of people based on ethnicity, gender, religion and other factors. The hate or contempt toward women has been increasing on social platforms. Misogynous content detection is especially challenging when textual and visual modalities are combined to form a single context, e.g., an overlay text embedded on top of an image, also known as meme. In this paper, we present a multimodal architecture that combines textual and visual features to detect misogynous memes. The proposed architecture is evaluated in the SemEval-2022 Task 5: MAMI - Multimedia Automatic Misogyny Identification challenge under the team name TIB-VA. We obtained the best result in the Task-B where the challenge is to classify whether a given document is misogynous and further identify the following sub-classes: shaming, stereotype, objectification, and violence.
Anthology ID:
2022.semeval-1.105
Volume:
Proceedings of the 16th International Workshop on Semantic Evaluation (SemEval-2022)
Month:
July
Year:
2022
Address:
Seattle, United States
Editors:
Guy Emerson, Natalie Schluter, Gabriel Stanovsky, Ritesh Kumar, Alexis Palmer, Nathan Schneider, Siddharth Singh, Shyam Ratan
Venue:
SemEval
SIG:
SIGLEX
Publisher:
Association for Computational Linguistics
Note:
Pages:
756–760
Language:
URL:
https://aclanthology.org/2022.semeval-1.105
DOI:
10.18653/v1/2022.semeval-1.105
Bibkey:
Cite (ACL):
Sherzod Hakimov, Gullal Singh Cheema, and Ralph Ewerth. 2022. TIB-VA at SemEval-2022 Task 5: A Multimodal Architecture for the Detection and Classification of Misogynous Memes. In Proceedings of the 16th International Workshop on Semantic Evaluation (SemEval-2022), pages 756–760, Seattle, United States. Association for Computational Linguistics.
Cite (Informal):
TIB-VA at SemEval-2022 Task 5: A Multimodal Architecture for the Detection and Classification of Misogynous Memes (Hakimov et al., SemEval 2022)
Copy Citation:
PDF:
https://aclanthology.org/2022.semeval-1.105.pdf
Video:
 https://aclanthology.org/2022.semeval-1.105.mp4
Code
 tibhannover/multimodal-misogyny-detection-mami-2022