Most existing benchmarks evaluate social bias in large language models (LLMs) using multiple-choice questions (MCQ) grounded in stereotypical contexts. While MCQ formats enable straightforward quantification, how LLMs manifest social bias in task-specific contexts and under different output constraints remains underexplored. We introduce SCENE, a benchmark for measuring the social bias shown by LLMs in scenario-completion tasks. Given short scenarios involving social groups, models must complete key elements of the situation through two tasks: (i) Character Specification, which asks models to assign social attributes to characters, and (ii) Plot Continuation, which asks models to select the most plausible narrative continuation. Combining templates adapted from prior benchmarks with newly collected templates through a university-wide competition, we construct a dataset of over 75,000 Korean instances. Experiments show that bias measurements are highly sensitive to task context and output constraints, even for identical underlying scenarios, suggesting that widely used MCQ-based evaluations may not fully capture certain manifestations of stereotypes. SCENE is publicly available at https://github.com/DSAIL-SKKU/SCENE-EMNLP-2026.

While prior studies have shown that gendered stereotypes can influence LLMs’ outputs, little attention has been paid to how gendered cues affect depression severity assessment in mental health contexts. In this paper, we systematically investigate the impact of gendered cues on LLM-based depression severity prediction, focusing on depressive disorder. To this end, we introduce a counterfactual evaluation framework, where we inject two types of gendered cues, explicit and implicit, into social media posts and analyze how they shift LLMs’ severity predictions. Our results reveal that gendered cues systematically shift LLMs’ severity predictions, increasing the risk of underestimating severe cases that may involve suicidal ideation or self-harm. To explore a possible mitigation direction, we propose FairDep, a symptom-grounded reasoning scaffold designed to reduce LLM sensitivity to gendered cues in depression severity prediction. By guiding models to focus on symptom-level evidence, FairDep improves prediction consistency under gender cue perturbations while preserving sensitivity to severe cases. Our findings show that gender cue sensitivity is not merely a fairness concern, but a safety concern in LLM-based mental health assessment, where even small prediction shifts can reduce sensitivity to severe cases involving suicidal ideation or self-harm. Our code and resources are publicly available.

Large language models (LLMs) are increasingly used for mental health applications, raising questions about whether they reflect established clinical knowledge. Clinical psychology has documented systematic cultural differences in the presentation of depression symptoms, with Western populations emphasizing emotional symptoms and many East Asian populations reporting more somatic symptoms. We evaluate whether general-purpose LLMs reproduce these clinically established cross-cultural patterns using prompts grounded in clinical descriptions of depression. We examine model responses under different cultural personas and languages.We find that LLMs struggle to reproduce expected cultural patterns when prompted in English. Prompting in major Eastern languages improves alignment in some configurations, suggesting that language cues partially activate cultural knowledge. However, model behavior remains dominated by a strong, culture-invariant hierarchy of depression symptoms that often overrides cultural cues, highlighting limitations in current LLMs for mental health applications.

As the explainability of mental disorder detection models has become important, symptom-based methods that predict disorders from identified symptoms have been widely utilized. However, since these approaches focused on the presence of symptoms, the context of symptoms can be often ignored, leading to missing important contextual information related to detecting mental disorders. Furthermore, the result of disorder detection can be vulnerable to errors that may occur in identifying symptoms. To address these issues, we propose a novel framework that detects mental disorders by leveraging symptoms and their context while mitigating potential errors in symptom identification. In this way, we propose to use large language models to effectively extract contextual information and introduce an uncertainty-aware decision fusion network that combines predictions of multiple models based on quantified uncertainty values. To evaluate the proposed method, we constructed a new Korean mental health dataset annotated by experts, named KoMOS. Experimental results demonstrate that the proposed model accurately detects mental disorders even in situations where symptom information is incomplete.

Discovering individuals’ suicidality on social media has become increasingly important. Many researchers have studied to detect suicidality by using a suicide dictionary. However, while prior work focused on matching a word in a post with a suicide dictionary without considering contexts, little attention has been paid to how the word can be associated with the suicide-related context. To address this problem, we propose a suicidality detection model based on a graph neural network to grasp the dynamic semantic information of the suicide vocabulary by learning the relations between a given post and words. The extensive evaluation demonstrates that the proposed model achieves higher performance than the state-of-the-art methods. We believe the proposed model has great utility in identifying the suicidality of individuals and hence preventing individuals from potential suicide risks at an early stage.