Koustuv Saha
Author directory2026
Belief Is All You Need: Signed Belief Graph Neural Networks for Topic Modeling in Conspiratorial Discourse
Soorya Ram Shimgekar | Abhay Goyal | Roy Ka-Wei Lee | Koustuv Saha | Pi Zonooz | Edson C Tandoc Jr | Navin Kumar
Proceedings of the Seventh Workshop on Natural Language Processing and Computational Social Science
Soorya Ram Shimgekar | Abhay Goyal | Roy Ka-Wei Lee | Koustuv Saha | Pi Zonooz | Edson C Tandoc Jr | Navin Kumar
Proceedings of the Seventh Workshop on Natural Language Processing and Computational Social Science
Conspiratorial discourse is increasingly present in online communication, yet how it is organized across discussion topics remains unclear. We analyze Singapore-based Telegram groups to examine how conspiratorial content appears within everyday conversations rather than isolated echo chambers. To better capture the structure of such discussions, we propose a two-stage framework for topic modeling tailored to conspiratorial posts. First, a RoBERTa-large classifier identifies conspiratorial messages (F1 = 0.866) using 2,000 expert-annotated examples. We then construct a graph where connections reflect textual similarity and conspiratorial stance. This graph is modeled using a Signed Belief Graph Neural Network (SiBeGNN), which learns message embeddings that distinguish conspiratorial from non-conspiratorial content. We apply hierarchical clustering on these embeddings to perform topic modeling over 553,648 Telegram messages, producing seven topic clusters: General Legal Topics, Medical Concerns, Media Discussions, Banking and Finance, Contradictions in Authority, Group Moderation, and General Discussions. Our method substantially outperforms standard embedding-based clustering approaches (cDBI = 8.38 vs. 13.60–67.27), with manual evaluation showing 88% inter-rater agreement in cluster interpretation. The results show that conspiratorial content appears across multiple everyday topics, including finance, law, and daily life, rather than forming isolated thematic communities.
Answer Bubbles: Information Exposure in AI-Mediated Search
Michelle Huang | Agam Goyal | Koustuv Saha | Eshwar Chandrasekharan
Findings of the Association for Computational Linguistics: EMNLP 2026
Michelle Huang | Agam Goyal | Koustuv Saha | Eshwar Chandrasekharan
Findings of the Association for Computational Linguistics: EMNLP 2026
Generative search systems are increasingly replacing link-based retrieval with AI-generated summaries, yet little is known about how these systems differ in sources, language, and fidelity to cited material. We examine responses to 11,000 real search queries across five systems—vanilla GPT, Search GPT, Perplexity Search with Grok, Google AI Overviews, and traditional Google Search—at three levels: source diversity, linguistic characterization of the generated summary, and source-summary fidelity. We find that generative search systems exhibit significant source-selection biases in their citations, favoring certain sources over others. Incorporating search also selectively attenuates epistemic markers, reducing hedging by up to 60% while preserving confidence language in the AI-generated summaries. At the same time, AI summaries further compound the citation biases: Wikipedia and longer sources are disproportionately overrepresented, whereas cited social media content and negatively framed sources are substantially underrepresented. Our findings highlight the potential for answer bubbles, in which identical queries yield structurally different information realities across systems, with implications for user trust, source visibility, and the transparency of AI-mediated information access.
TUX: Measuring Human–AI Tacit Understanding
Yueshen Li | Hanyi Min | Vedant Das Swain | Koustuv Saha
Findings of the Association for Computational Linguistics: EMNLP 2026
Yueshen Li | Hanyi Min | Vedant Das Swain | Koustuv Saha
Findings of the Association for Computational Linguistics: EMNLP 2026
As large language models (LLMs) increasingly act as collaborative partners, human–AI alignment is often evaluated through explicit task success, accuracy, or reward optimization. Yet many collaborative settings depend on tacit understanding: whether an agent can align with a human’s evaluative stance or representational priors without clear objectives, communication, or feedback. To study this capacity, we develop a spectrum-placement task inspired by the social party game Wavelength, in which humans and agents independently place concepts along subjective spectra. We operationalize the Tacit Understanding Index (TUX) as a pairwise behavioral measure of similarity between human and agent judgments, and evaluate it with 241 human participants and 200 profile-conditioned LLM agents across four models. We find that nearest human–agent pairs in trait space achieve significantly higher TUX, suggesting that tacit alignment is associated with person-level characteristics rather than reflecting only random similarity. Regression analyses show that TUX becomes more explainable as predictor sets become richer, with individual traits, decision-making styles, and confidence improving over aggregate trait-distance baselines. These findings suggest that TUX provides a measurable behavioral signal of human–LLM tacit understanding, while revealing the limits of profile-based conditioning for capturing deeper representational alignment.
RubRIX: Rubric-Driven Risk Mitigation in Caregiver-AI Interactions
Drishti Goel | Jeongah Lee | Qiuyue Zhong | Violeta J. Rodriguez | Daniel S. Brown | Ravi Karkar | Dong Whi Yoo | Koustuv Saha
Findings of the Association for Computational Linguistics: ACL 2026
Drishti Goel | Jeongah Lee | Qiuyue Zhong | Violeta J. Rodriguez | Daniel S. Brown | Ravi Karkar | Dong Whi Yoo | Koustuv Saha
Findings of the Association for Computational Linguistics: ACL 2026
Caregivers seeking AI-mediated support express complex needs—information-seeking, emotional validation, and distress cues—that warrant careful evaluation of response safety and appropriateness. Existing AI evaluation frameworks, primarily focused on general risks (toxicity, hallucinations, policy violations, etc) may not adequately capture the nuanced risks of LLM-responses in caregiving-contexts. We introduce RubRIX (Rubric-based Risk Index), a theory-driven, clinician-validated framework for evaluating risks in LLM caregiving responses. Grounded in the Elements of an Ethic of Care, RubRIX operationalizes five empirically-derived risk dimensions: Inattention, Bias Stigma, Information Inaccuracy, Uncritical Affirmation, and Epistemic Arrogance. We evaluate six state-of-the-art LLMs on over 20,000 caregiver queries from Reddit and ALZConnected. Rubric-guided refinement consistently reduced risk-components by 45-98% after one iteration across models. This work contributes a methodological approach for developing domain-sensitive, user-centered evaluation frameworks for high-burden contexts. Our findings highlight the importance of domain-sensitive, interactional risk evaluation for the responsible deployment of LLMs in caregiving support contexts. We release benchmark datasets to enable future research on contextual risk evaluation in AI-mediated support.
The Hidden Toll of Social Media News on Psychosocial Wellbeing
Olivia Pal | Agam Goyal | Eshwar Chandrasekharan | Koustuv Saha
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Olivia Pal | Agam Goyal | Eshwar Chandrasekharan | Koustuv Saha
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
News consumption on social media has become ubiquitous, yet how different forms of engagement shape psychosocial outcomes remains unclear. To address this gap, we leveraged a large-scale dataset of ~26M posts and ~45M comments on the Bluesky platform and conducted a quasi-experimental study, matching 81,345 Treated users exposed to News feeds with 83,711 Control users using stratified propensity score analysis. We examined psychosocial wellbeing in terms of affective, behavioral, and cognitive outcomes. Our findings reveal that news engagement produces systematic trade-offs: increased depression, stress, and anxiety, yet decreased loneliness and increased social interaction on the platform. Regression models reveal that News feed bookmarking is associated with greater psychosocial deterioration compared to commenting or quoting, with magnitude differences exceeding tenfold. These per-engagement effects accumulate with repeated exposure, showing significant psychosocial impacts. Our work extends theories of news effects beyond crisis-centric frameworks by demonstrating that routine consumption creates distinct psychological dynamics depending on engagement type, and bears implications for tools and interventions for mitigating the psychosocial costs of news consumption on social media.
VASTU: Language Models Struggle to Recognize Online Community Values
Agam Goyal | Xianyang Zhan | Charlotte Lambert | Koustuv Saha | Eshwar Chandrasekharan
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Agam Goyal | Xianyang Zhan | Charlotte Lambert | Koustuv Saha | Eshwar Chandrasekharan
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Online communities develop distinct norms for content they collectively value, yet it remains unclear whether current language models can recognize locally valued contributions in context. We formalize this as community-conditioned preference prediction and introduce Vastu (Value-Aware Social Tuning), a benchmark of 75,000 Reddit comments from 15 communities spanning Gaming, Science, Q&A, Advice, and Politics. We evaluate four model families—prompted LLMs, LoRA-adapted SLMs, supervised encoders, and feature-based classifiers—across global, local, and context-conditioned settings. Our central finding is that parametric adaptation consistently outperforms prompting: supervised encoders reach 0.74 AUROC and fine-tuned SLMs 0.64–0.71, while the best prompted result is only 0.62. This gap is not merely quantitative—vanilla prompting yields over 80% false-negative rates, systematically discarding content communities actually value. Conversational context narrows but does not close this divide. Together, these results suggest that local preference recognition requires community-specific training signal, not just better prompting. Our work supports future research on community-aware reward modeling, feed curation, and positive moderation.
LLUMI: Improving LLM Writing Assistance for Mental Health Support with Online Community Feedback
Jenny Jiwon Kim | Maya Ajit | Sherry Gong | Soorya Ram Shimgekar | Dong Whi Yoo | Eshwar Chandrasekharan | Koustuv Saha
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Jenny Jiwon Kim | Maya Ajit | Sherry Gong | Soorya Ram Shimgekar | Dong Whi Yoo | Eshwar Chandrasekharan | Koustuv Saha
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Large language models (LLMs) show promise in generating supportive responses for mental health queries, but improving their usefulness, empathy, and safety often requires substantial compute, expert input, and labeled data. At the same time, deploying proprietary, cloud-based models for mental health-related interactions raises important privacy and data-governance concerns, given the sensitivities. To address this challenge, we introduce LLUMI setup that can be hosted in-house within protected environments. LLUMI consists of two complementary components: a generation model (GM), which drafts supportive responses to mental health queries, and an improvement model (IM), which revises an initial human-crafted response. We leverage feedback signals from Reddit mental health communities, using community endorsement patterns such as upvotes and downvotes to construct chosen–rejected response pairs for Supervised Fine Tuning (SFT) and Direct Preference Optimization (DPO). We further align LLUMI using human evaluation across five dimensions: readability, empathy, connection, actionability, and safety. Our results show that, despite relying on smaller open-source models rather than proprietary cloud-based GPT models, LLUMI achieves comparable performance across linguistic analyses and human evaluations. These findings suggest that open-source models, when trained with community-derived preference signals, can support high-quality mental health support assistance while offering a more privacy-preserving alternative for sensitive support contexts.
AI Psychosis: Does Conversational AI Amplify Delusion-Related Language?
Soorya Ram Shimgekar | Vipin Gunda | Jenny Jiwon Kim | Violeta J. Rodriguez | Hari Sundaram | Koustuv Saha
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Soorya Ram Shimgekar | Vipin Gunda | Jenny Jiwon Kim | Violeta J. Rodriguez | Hari Sundaram | Koustuv Saha
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Conversational AI systems are increasingly used for personal reflection and emotional disclosure, raising concerns about their effects on vulnerable users. Recent anecdotal reports suggest that prolonged interactions with conversational AI may reinforce delusional thinking—a phenomenon sometimes described as AI Psychosis. However, systematic empirical evidence on this phenomenon remains limited. In this work, we examine how delusion-related language evolves during multi-turn interactions with conversational AI. Using Reddit data, we construct simulated users (SimUsers) from users’ longitudinal posting histories and generate extended conversations with three model families (GPT, LLaMA, and Qwen). We develop DelusionScore, a linguistic measure that quantifies the intensity of delusion-related language across conversational turns. We find that SimUsers derived from users with prior delusion-related discourse (Treatment) exhibit progressively increasing DelusionScore trajectories, while those derived from users without such discourse (Control) remain stable or decline. We further find that this amplification varies across themes, with reality skepticism and compulsive reasoning showing the strongest increases. Finally, conditioning AI responses on the current DelusionScore substantially reduces these trajectories. These findings provide empirical evidence that conversational AI interactions can amplify delusion-related language over extended dialogue and underscore the importance of state-aware safety mechanisms to mitigate such risks.
2025
SLM-Mod: Small Language Models Surpass LLMs at Content Moderation
Xianyang Zhan | Agam Goyal | Yilun Chen | Eshwar Chandrasekharan | Koustuv Saha
Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers)
Xianyang Zhan | Agam Goyal | Yilun Chen | Eshwar Chandrasekharan | Koustuv Saha
Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers)
Large language models (LLMs) have shown promise in many natural language understanding tasks, including content moderation. However, these models can be expensive to query in real-time and do not allow for a community-specific approach to content moderation. To address these challenges, we explore the use of open-source small language models (SLMs) for community-specific content moderation tasks. We fine-tune and evaluate SLMs (less than 15B parameters) by comparing their performance against much larger open- and closed-sourced models in both a zero-shot and few-shot setting. Using 150K comments from 15 popular Reddit communities, we find that SLMs outperform zero-shot LLMs at content moderation-11.5% higher accuracy and 25.7% higher recall on average across all communities. Moreover, few-shot in-context learning leads to only a marginal increase in the performance of LLMs, still lacking compared to SLMs. We further show the promise of cross-community content moderation, which has implications for new communities and the development of cross-platform moderation techniques. Finally, we outline directions for future work on language model based content moderation.
MoMoE: Mixture of Moderation Experts Framework for AI-Assisted Online Governance
Agam Goyal | Xianyang Zhan | Yilun Chen | Koustuv Saha | Eshwar Chandrasekharan
Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing
Agam Goyal | Xianyang Zhan | Yilun Chen | Koustuv Saha | Eshwar Chandrasekharan
Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing
Large language models (LLMs) have shown great potential in flagging harmful content in online communities. Yet, existing approaches for moderation require a separate model for every community and are opaque in their decision-making, limiting real-world adoption. We introduce Mixture of Moderation Experts (MoMoE), a modular, cross-community framework that adds post-hoc explanations to enable scalable content moderation. MoMoE orchestrates four operators—Allocate, Predict, Aggregate, Explain—and is instantiated as seven community-specialized experts (MoMoE-Community) and five norm-violation experts (MoMoE-NormVio). On 30 unseen subreddits, the best variants obtain Micro-F1 scores of 0.72 and 0.67, respectively, matching or surpassing strong fine-tuned baselines while consistently producing concise and reliable explanations. Although community-specialized experts deliver the highest peak accuracy, norm-violation experts provide steadier performance across domains. These findings show that MoMoE yields scalable, transparent moderation without needing per-community fine-tuning. More broadly, they suggest that lightweight, explainable expert ensembles can guide future NLP and HCI research on trustworthy human-AI governance of online communities.
Search
Fix author
Co-authors
- Eshwar Chandrasekharan 6
- Agam Goyal 5
- Soorya Ram Shimgekar 3
- Xianyang Zhan 3
- Yilun Chen 2
- Jenny Jiwon Kim 2
- Violeta J. Rodriguez 2
- Dong Whi Yoo 2
- Maya Ajit 1
- Daniel S. Brown 1
- Drishti Goel 1
- Sherry Gong 1
- Abhay Goyal 1
- Vipin Gunda 1
- Michelle Huang 1
- Ravi Karkar 1
- Navin Kumar 1
- Charlotte Lambert 1
- Jeongah Lee 1
- Roy Ka-Wei Lee 1
- Yueshen Li 1
- Hanyi Min 1
- Olivia Pal 1
- Hari Sundaram 1
- Vedant Das Swain 1
- Edson C Tandoc Jr 1
- Qiuyue Zhong 1
- Pi Zonooz 1