2024
pdf
bib
abs
Language Models (Mostly) Do Not Consider Emotion Triggers When Predicting Emotion
Smriti Singh
|
Cornelia Caragea
|
Junyi Jessy Li
Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 2: Short Papers)
Situations and events evoke emotions in humans, but to what extent do they inform the prediction of emotion detection models? This work investigates how well human-annotated emotion triggers correlate with features that models deemed salient in their prediction of emotions. First, we introduce a novel dataset EmoTrigger, consisting of 900 social media posts sourced from three different datasets; these were annotated by experts for emotion triggers with high agreement. Using EmoTrigger, we evaluate the ability of large language models (LLMs) to identify emotion triggers, and conduct a comparative analysis of the features considered important for these tasks between LLMs and fine-tuned models. Our analysis reveals that emotion triggers are largely not considered salient features for emotion prediction models, instead there is intricate interplay between various features and the task of emotion detection.
2023
pdf
bib
abs
“Female Astronaut: Because sandwiches won’t make themselves up there”: Towards Multimodal misogyny detection in memes
Smriti Singh
|
Amritha Haridasan
|
Raymond Mooney
The 7th Workshop on Online Abuse and Harms (WOAH)
A rise in the circulation of memes has led to the spread of a new form of multimodal hateful content. Unfortunately, the degree of hate women receive on the internet is disproportionately skewed against them. This, combined with the fact that multimodal misogyny is more challenging to detect as opposed to traditional text-based misogyny, signifies that the task of identifying misogynistic memes online is one of utmost importance. To this end, the MAMI dataset was released, consisting of 12000 memes annotated for misogyny and four sub-classes of misogyny - shame, objectification, violence and stereotype. While this balanced dataset is widely cited, we find that the task itself remains largely unsolved. Thus, in our work, we investigate the performance of multiple models in an effort to analyse whether domain specific pretraining helps model performance. We also investigate why even state of the art models find this task so challenging, and whether domain-specific pretraining can help. Our results show that pretraining BERT on hateful memes and leveraging an attention based approach with ViT outperforms state of the art models by more than 10%. Further, we provide insight into why these models may be struggling with this task with an extensive qualitative analysis of random samples from the test set.
2021
pdf
bib
abs
“Hold on honey, men at work”: A semi-supervised approach to detecting sexism in sitcoms
Smriti Singh
|
Tanvi Anand
|
Arijit Ghosh Chowdhury
|
Zeerak Waseem
Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing: Student Research Workshop
Television shows play an important role inpropagating societal norms. Owing to the popularity of the situational comedy (sitcom) genre, it contributes significantly to the over-all development of society. In an effort to analyze the content of television shows belong-ing to this genre, we present a dataset of dialogue turns from popular sitcoms annotated for the presence of sexist remarks. We train a text classification model to detect sexism using domain adaptive learning. We apply the model to our dataset to analyze the evolution of sexist content over the years. We propose a domain-specific semi-supervised architecture for the aforementioned detection of sexism. Through extensive experiments, we show that our model often yields better classification performance over generic deep learn-ing based sentence classification that does not employ domain-specific training. We find that while sexism decreases over time on average,the proportion of sexist dialogue for the most sexist sitcom actually increases. A quantitative analysis along with a detailed error analysis presents the case for our proposed methodology
2014
pdf
bib
HinMA: Distributed Morphology based Hindi Morphological Analyzer
Ankit Bahuguna
|
Lavita Talukdar
|
Pushpak Bhattacharyya
|
Smriti Singh
Proceedings of the 11th International Conference on Natural Language Processing
2012
pdf
bib
Noun Group and Verb Group Identification for Hindi
Smriti Singh
|
Om P. Damani
|
Vaijayanthi M. Sarma
Proceedings of COLING 2012
2011
pdf
bib
Verbal Inflection in Hindi: A Distributed Morphology Approach
Smriti Singh
|
Vaijayanthi M. Sarma
Proceedings of the 25th Pacific Asia Conference on Language, Information and Computation
2007
pdf
bib
Hindi generation from interlingua
Smriti Singh
|
Mrugank Dalal
|
Vishal Vachhani
|
Pushpak Bhattacharyya
|
Om P. Damani
Proceedings of Machine Translation Summit XI: Papers
2006
pdf
bib
Morphological Richness Offsets Resource Demand – Experiences in Constructing a POS Tagger for Hindi
Smriti Singh
|
Kuhoo Gupta
|
Manish Shrivastava
|
Pushpak Bhattacharyya
Proceedings of the COLING/ACL 2006 Main Conference Poster Sessions