Proceedings of the 21st International Conference on Natural Language Processing (ICON): Shared Task on Decoding Fake Narratives in Spreading Hateful Stories (Faux-Hate)

Shankar Biradar, Kasu Sai Kartheek Reddy, Sunil Saumya, Md. Shad Akhtar (Editors)


Anthology ID:
2024.icon-fauxhate
Month:
December
Year:
2024
Address:
AU-KBC Research Centre, Chennai, India
Venue:
ICON
SIG:
SIGLEX
Publisher:
NLP Association of India (NLPAI)
URL:
https://aclanthology.org/2024.icon-fauxhate/
DOI:
Bib Export formats:
BibTeX MODS XML EndNote

pdf bib
Proceedings of the 21st International Conference on Natural Language Processing (ICON): Shared Task on Decoding Fake Narratives in Spreading Hateful Stories (Faux-Hate)
Shankar Biradar | Kasu Sai Kartheek Reddy | Sunil Saumya | Md. Shad Akhtar

pdf bib
Proceedings of the 21st International Conference on Natural Language Processing (ICON): Shared Task on Decoding Fake Narratives in Spreading Hateful Stories (Faux-Hate)
Shankar Biradar | Kasu Sai Kartheek Reddy | Sunil Saumya | Md. Shad Akhtar

The rapid expansion of social media has led toan increase in code-mixed content, presentingsignificant challenges in the effective detectionof hate speech and fake narratives. To advanceresearch in this area, a shared task titled De-coding Fake Narratives in Spreading HatefulStories (Faux-Hate) was organized as part ofICON 2024. This paper introduces a multi-task learning model designed to classify Hindi-English code-mixed tweets into two distinct cat-egories: hate speech and false content. The pro-posed framework utilizes fastText embeddingsto create a shared feature space that adeptly cap-tures the semantic and syntactic intricacies ofcode-mixed text, including transliterated termsand out-of-vocabulary words. These sharedembeddings are then processed through twoindependent Support Vector Machine (SVM)classifiers, each specifically tailored for oneof the classification tasks. Our team, secured10th place among the participating teams, asevaluated by the organizers based on Macro F1scores.

pdf bib
Unpacking Faux-Hate: Addressing Faux-Hate Detection and Severity Prediction in Code-Mixed Hinglish Text with HingRoBERTa and Class Weighting Techniques
Ashweta A. Fondekar | Milind M. Shivolkar | Jyoti D. Pawar

The proliferation of hate speech and fake narra-tives on social media poses significant societalchallenges, especially in multilingual and code-mixed contexts. This paper presents our systemsubmitted to the ICON 2024 shared task onDecoding Fake Narratives in Spreading Hate-ful Stories (Faux-Hate). We tackle the prob-lem of Faux-Hate Detection, which involvesdetecting fake narratives and hate speech incode-mixed Hinglish text. Leveraging Hin-gRoBERTa, a pre-trained transformer modelfine-tuned on Hinglish datasets, we addresstwo sub-tasks: Binary Faux-Hate Detection andTarget and Severity Prediction. Through the in-troduction of class weighting techniques andthe optimization of a multi-task learning ap-proach, we demonstrate improved performancein identifying hate and fake speech, as well asin classifying their target and severity. Thisresearch contributes to a scalable and efficientframework for addressing complex real-worldtext processing challenges.

pdf bib
Decoding Fake Narratives in Spreading Hateful Stories: A Dual-Head RoBERTa Model with Multi-Task Learning
Yash Bhaskar | Sankalp Bahad | Parameswari Krishnamurthy

Social media platforms, while enabling globalconnectivity, have become hubs for the rapidspread of harmful content, including hatespeech and fake narratives (Davidson et al.,2017; Shu et al., 2017). The Faux-Hateshared task focuses on detecting a specific phe-nomenon: the generation of hate speech drivenby fake narratives, termed Faux-Hate. Partici-pants are challenged to identify such instancesin code-mixed Hindi-English social media text.This paper describes our system developed forthe shared task, addressing two primary sub-tasks: (a) Binary Faux-Hate detection, involv-ing fake and hate speech classification, and(b) Target and Severity prediction, categoriz-ing the intended target and severity of hate-ful content. Our approach combines advancednatural language processing techniques withdomain-specific pretraining to enhance perfor-mance across both tasks. The system achievedcompetitive results, demonstrating the efficacyof leveraging multi-task learning for this com-plex problem.

pdf bib
Challenges and Insights in Identifying Hate Speech and Fake News on Social Media
Shanthi Murugan | Arthi R | Boomika E | Jeyanth S | Kaviyarasu S

Social media has transformed communication, but it has also brought abouta number of serious problems, most notablythe proliferation of hate speech and falseinformation. hate-related conversations arefrequently fueled by misleading narratives.We address this issue by building a multiclassclassification model trained on Faux HateMulti-Label Dataset (Biradar et al. 2024)which consists of hateful remarks that arefraudulent and have a code mix of Hindi andEnglish. Model has been built to classifySeverity (Low, Medium, High) and Target(Individual, Organization, Religion) on thedataset. Performance of the model isevaluated on test dataset achieved varyingscored for each. For Severity model achieves74%, for Target model achieves 74%. Thelimitations and performance issues of themodel has been understood and wellexplained.

pdf bib
Detecting Hate Speech and Fake Narratives in Code-Mixed Hinglish Social Media Text
Advaitha Vetagiri | Partha Pakray

The increasing prevalence of hate speech and fake narratives on social media platforms posessignificant societal challenges. This study ad-dresses these issues through the developmentof robust machine learning models for twotasks: (1) detecting hate speech and fake nar-ratives (Task A) and (2) predicting the targetand severity of hateful content (Task B) incode-mixed Hindi-English text. We proposefour separate CNN-BiLSTM models tailoredfor each subtask. The models were evaluatedusing validation and 5-fold cross-validationdatasets, achieving F1-scores of 74% and 79%for hate and fake detection, respectively, and63% and 54% for target and severity predic-tion and achieved 65% and 57% for testingresults. The results highlight the models’ effec-tiveness in handling the nuances of code-mixedtext while underscoring the challenges of under-represented classes. This work contributes tothe ongoing effort to develop automated toolsfor detecting and mitigating harmful contentonline, paving the way for safer and more in-clusive digital spaces.

pdf bib
Transformer-driven Multi-task Learning for Fake and Hateful Content Detection
Asha Hegde | H L Shashirekha

Social media has revolutionized communica-tion these days in addition to facilitating thespread of fake and hate content. While fakecontent is the manipulation of facts by disin-formation, hate content is textual violence ordiscrimination targeting a group or an individ-ual. Fake narratives have the potential to spreadhate content making people aggressive or hurt-ing the sentiments of an individual or a group.Further, false narratives often dominate discus-sions on sensitive topics, amplifying harmfulmessages contributing to the rise of hate speech.Hence, understanding the relationship betweenhate speech driven by fake narratives is cru-cial in this digital age making it necessary todevelop automatic tools to identify fake andhate content. In this direction, Decoding FakeNarratives in Spreading Hateful Stories (Faux-Hate) - a shared task organized at the Inter-national Conference on Natural Language Pro-cessing (ICON) 2024, invites researchers totackle both fake and hate detection in socialmedia comments, with additional emphasis onidentifying the target and severity of hatefulspeech. The shared task consists of two sub-tasks - Task A (Identifying fake and hate con-tent) and Task B (Identifying the target andseverity of hateful speech). In this paper, we -team MUCS, describe the models proposed toaddress the challenges of this shared task. Wepropose two models: i) Hing_MTL - a Multi-task Learning (MTL) model implemented us-ing pre-trained Hinglish Bidirectional EncoderRepresentations from Transformers (Hinglish-BERT), and ii) Ensemble_MTL - a MTL modelimplemented by ensembling two pre-trainedmodels (HinglishBERT, and Multilingual Dis-tiled version of BERT (MDistilBERT)), to de-tect fake and hate content and identify the targetand severity of hateful speech. Ensemble_MTLmodel outperformed Hing_MTL model withmacro F1 scores of 0.7589 and 0.5746 for TaskA and Task B respectively, securing 6th placein both subtasks.

pdf bib
Rejected Cookies @ Decoding Faux-Hate: Predicting Fake Narratives and Hateful Content
Joel D Joy | Naman Srivastava

This paper reports the results of our team for theICON 2024 shared task Decoding Fake Narra-tives in Spreading Hateful Stories (Faux-Hate).The task aims at classifying tweets in a multi-label and multi-class framework. It comprisestwo subtasks: (A) Binary Faux-Hate Detec-tion, which involves predicting whether a tweetis fake (1/0) and/or hate speech (1/0, and (B)Target and Severity Prediction, which cate-gorizes tweets based on their target (Individ-ual, Organization, Religion) and severity (Low,Medium, High). We evaluated Machine Learn-ing (ML) approaches, including Logistic Re-gression, Support Vector Machines (SVM), andRandom Forest; Deep Learning (DL) methods,such as Artificial Neural Networks (ANN) andBidirectional Encoder Representations fromTransformers (BERT); and innovative quantumhybrid models, like Hybrid Quantum NeuralNetworks (HQNN), for identifying and classi-fying tweets across these subtasks. Our exper-iments trained and compared multiple modelarchitectures to assess their comparative per-formance and detection capabilities in these di-verse modeling strategies.The best-performingmodels achieved F1 scores of 0.72, 0.76, 0.64,and 0.54 for the respective labels Hate, Fake,Target and Severity. We have open-sourced ourimplementation code for both tasks on Github1 .

pdf bib
A Machine Learning Framework for Detecting Hate Speech and Fake Narratives in Hindi-English Tweets
R.n. Yadawad | Sunil Saumya | K.n. Nivedh | Siddhaling S. Padanur | Sudev Basti

This paper presents a novel system developed for the Faux-Hate Shared Task at ICON2024, addressing the detection of hate speechand fake narratives within Hindi-English code-mixed social media data. Our approach com-bines advanced text preprocessing, TF-IDFvectorization, and Random Forest classifiersto identify harmful content, while employingSMOTE to address class imbalance. By lever-aging ensemble learning and feature engineer-ing, our system demonstrates robust perfor-mance in detecting hateful and fake content,classifying targets, and evaluating the sever-ity of hate speech. The results underscore thepotential for real-world applications, such asmoderating online platforms and identifyingharmful narratives. Furthermore, we highlightethical considerations for deploying such tools,emphasizing responsible use in sensitive do-mains, thereby advancing research in multilin-gual hate speech detection and online abusemitigation.

pdf bib
Faux-Hate Multitask Framework for Misinformation and Hate Speech Detection in Code-Mixed Languages
Sunil Gopal C V | Sudhan S | Shreyas Gutti Srinivas | Sushanth R | Abhilash C B

The Faux-Hate task looks at two big issues:misinformation and hate speech. It focuses onHindi-English social media posts. This papershares our methods for both parts of the task.For Task A, we built a special model based onXLM-RoBERTa. It has features that help usspot both fake news and hate speech at the sametime. For Task B, we wanted to identify whothe hate is aimed at (like individuals or groups)and how severe it is (high, medium, low). So,we added different tools to our model for thiskind of sorting. To get ready for all this, wecarefully cleaned the data, especially dealingwith mixing languages and different spellings.In Task A, our results show that our model canclearly tell the difference between fake and realstories, as well as between hate and non-hatespeech. For Task B, it does a good job withidentifying targets and severity levels, givingstrong predictions for multiple labels. Overall,these results show that cross-lingual models,combined with specific tweaks, can really helptackle complex text classification in languageswith fewer resources.

pdf bib
Multi-Task Learning for Faux-Hate Detection in Hindi-English Code-Mixed Text
Hitesh N P | D Ankith | Poornachandra A N | Abhilash C B

The prevalence of harmful internet content is on the rise, especially among young people. Thismakes social media sites breeding grounds forhate speech and negativity even though theirpurpose is to create connections. The study pro-poses a multi-task learning model for the iden-tification and analysis of harmful social mediacontent. This classifies the text into fake/realand hate/non-hate categories and further identi-fies the target and severity of the harmful con-tent. The proposed model showed significantimprovements in performance with training ontransliterated data as compared to code-mixeddata. It ranked 2nd and 3rd in the ICON 2024Faux-Hate Shared Task and the performanceshave made it very effective against harmful con-tent.

pdf bib
LoRA adapter weight tuning with multi-task learning for Faux-Hate detection
Abhinandan Onajol | Varun Gani | Praneeta Marakatti | Bhakti Malwankar | Shankar Biradar

Detecting misinformation and harmful language in bilingual texts, particularly those com-bining Hindi and English, poses considerabledifficulties. The intricacies of mixed-languagecontent and limited available resources compli-cate this task even more. The proposed workfocuses on unraveling deceptive stories thatpropagate hate. We have developed an inno-vative attention-weight-tuned LoRA Adopter-based model for such Faux-Hate content de-tection. This work is conducted as a partof the ICON 2024 shared task on DecodingFake narratives in spreading Hateful stories.The LoRA-enhanced architecture secured 13thplace among the participating teams for TaskA.

pdf bib
Shared Feature-Based Multitask Model for Faux-Hate Classification in Code-Mixed Text
Sanjana Kavatagi | Rashmi Rachh | Prakul Hiremath

In recent years, the rise of harmful narratives online has highlighted the need for advancedhate speech detection models. One emergingchallenge is the phenomenon of Faux Hate, anew type of hate speech that originates fromthe intersection of fake narratives and hatespeech. Faux Hate occurs when fabricatedclaims fuel the generation of hateful language,often blurring the line between misinforma-tion and malicious intent. Identifying suchspeech becomes especially difficult when thefake claim itself is not immediately apparent.This paper provides an overview of a sharedtask competition focused on detecting FauxHate, where participants were tasked with de-veloping methodologies to identify this nu-anced form of harmful speech.