Codewithzichao@DravidianLangTech-EACL2021: Exploring Multimodal Transformers for Meme Classification in Tamil Language

Zichao Li


Abstract
This paper describes our submission to shared task on Meme Classification for Tamil Language. To address this task, we explore a multimodal transformer for meme classification in Tamil language. According to the characteristics of the image and text, we use different pretrained models to encode the image and text so as to get better representations of the image and text respectively. Besides, we design a multimodal attention layer to make the text and corresponding image interact fully with each other based on cross attention. Our model achieved 0.55 weighted average F1 score and ranked first in this task.
Anthology ID:
2021.dravidianlangtech-1.52
Volume:
Proceedings of the First Workshop on Speech and Language Technologies for Dravidian Languages
Month:
April
Year:
2021
Address:
Kyiv
Venues:
DravidianLangTech | EACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
352–356
Language:
URL:
https://aclanthology.org/2021.dravidianlangtech-1.52
DOI:
Bibkey:
Cite (ACL):
Zichao Li. 2021. Codewithzichao@DravidianLangTech-EACL2021: Exploring Multimodal Transformers for Meme Classification in Tamil Language. In Proceedings of the First Workshop on Speech and Language Technologies for Dravidian Languages, pages 352–356, Kyiv. Association for Computational Linguistics.
Cite (Informal):
Codewithzichao@DravidianLangTech-EACL2021: Exploring Multimodal Transformers for Meme Classification in Tamil Language (Li, DravidianLangTech 2021)
Copy Citation:
PDF:
https://aclanthology.org/2021.dravidianlangtech-1.52.pdf
Software:
 2021.dravidianlangtech-1.52.Software.zip