<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="pranesh-2022-exploring">
    <titleInfo>
        <title>Exploring Multimodal Features and Fusion Strategies for Analyzing Disaster Tweets</title>
    </titleInfo>
    <name type="personal">
        <namePart type="given">Raj</namePart>
        <namePart type="family">Pranesh</namePart>
        <role>
            <roleTerm authority="marcrelator" type="text">author</roleTerm>
        </role>
    </name>
    <originInfo>
        <dateIssued>2022-10</dateIssued>
    </originInfo>
    <typeOfResource>text</typeOfResource>
    <relatedItem type="host">
        <titleInfo>
            <title>Proceedings of the Eighth Workshop on Noisy User-generated Text (W-NUT 2022)</title>
        </titleInfo>
        <originInfo>
            <publisher>Association for Computational Linguistics</publisher>
            <place>
                <placeTerm type="text">Gyeongju, Republic of Korea</placeTerm>
            </place>
        </originInfo>
        <genre authority="marcgt">conference publication</genre>
    </relatedItem>
    <abstract>Social media platforms, such as Twitter, often provide firsthand news during the outbreak of a crisis. It is extremely essential to process these facts quickly to plan the response efforts for minimal loss. Therefore, in this paper, we present an analysis of various multimodal feature fusion techniques to analyze and classify disaster tweets into multiple crisis events via transfer learning. In our study, we utilized three image models pre-trained on ImageNet dataset and three fine-tuned language models to learn the visual and textual features of the data and combine them to make predictions. We have presented a systematic analysis of multiple intra-modal and cross-modal fusion strategies and their effect on the performance of the multimodal disaster classification system. In our experiment, we used 8,242 disaster tweets, each comprising image, and text data with five disaster event classes. The results show that the multimodal with transformer-attention mechanism and factorized bilinear pooling (FBP) for intra-modal and cross-modal feature fusion respectively achieved the best performance.</abstract>
    <identifier type="citekey">pranesh-2022-exploring</identifier>
    <location>
        <url>https://aclanthology.org/2022.wnut-1.6</url>
    </location>
    <part>
        <date>2022-10</date>
        <extent unit="page">
            <start>62</start>
            <end>68</end>
        </extent>
    </part>
</mods>
</modsCollection>

%0 Conference Proceedings
%T Exploring Multimodal Features and Fusion Strategies for Analyzing Disaster Tweets
%A Pranesh, Raj
%S Proceedings of the Eighth Workshop on Noisy User-generated Text (W-NUT 2022)
%D 2022
%8 October
%I Association for Computational Linguistics
%C Gyeongju, Republic of Korea
%F pranesh-2022-exploring
%X Social media platforms, such as Twitter, often provide firsthand news during the outbreak of a crisis. It is extremely essential to process these facts quickly to plan the response efforts for minimal loss. Therefore, in this paper, we present an analysis of various multimodal feature fusion techniques to analyze and classify disaster tweets into multiple crisis events via transfer learning. In our study, we utilized three image models pre-trained on ImageNet dataset and three fine-tuned language models to learn the visual and textual features of the data and combine them to make predictions. We have presented a systematic analysis of multiple intra-modal and cross-modal fusion strategies and their effect on the performance of the multimodal disaster classification system. In our experiment, we used 8,242 disaster tweets, each comprising image, and text data with five disaster event classes. The results show that the multimodal with transformer-attention mechanism and factorized bilinear pooling (FBP) for intra-modal and cross-modal feature fusion respectively achieved the best performance.
%U https://aclanthology.org/2022.wnut-1.6
%P 62-68

Markdown (Informal)

[Exploring Multimodal Features and Fusion Strategies for Analyzing Disaster Tweets](https://aclanthology.org/2022.wnut-1.6) (Pranesh, WNUT 2022)

• Exploring Multimodal Features and Fusion Strategies for Analyzing Disaster Tweets (Pranesh, WNUT 2022)

ACL

• Raj Pranesh. 2022. Exploring Multimodal Features and Fusion Strategies for Analyzing Disaster Tweets. In Proceedings of the Eighth Workshop on Noisy User-generated Text (W-NUT 2022), pages 62–68, Gyeongju, Republic of Korea. Association for Computational Linguistics.