Workshop on Multimodal, Multilingual, and Multicultural Mental Health and Psychotherapy (2026)

1 volumes · 16 papers (0% with DOI) · 67 unique authors (37% verified with ORCID)


up

pdf (full)
bib (full)
Proceedings of the 1st Workshop on Multimodal, Multilingual, and Multicultural Mental Health and Psychotherapy (MultiPsyche)

LLM-based AI therapists largely assume cooperative clients, despite real-world therapeutic interactions often involving resistant behaviors: withdrawn, deflecting, denying, or hostile. We propose a resistance-aware AI therapist combining (i) a multi-channel resistance detector that classifies each client turn into one of five types (R0–R4), (ii) a stabilized policy selector that maps a detected resistance to an evidence-based therapeutic strategy, and (iii) a strategy-grounded response generator. Across 800 simulated sessions over ten DSM-5 disorder profiles, the detector reaches 85% accuracy and 85.5% macro F1. Within the simulated evaluation, the adaptive policy achieves resistance-reduction rates of 70–100% across resistance types, whereas each static policy yields zero reduction for at least one type. The adaptive policy also maintains resistance-reduction performance across ten simulated DSM-5 disorder profiles. Our dataset and implementation are fully open-sourced.
Reference-free translation audits have little to work with when a survey answer is only one or two words long. A translation may be reasonable and still be nearly impossible to reconnect to its source among thousands of similar answers. We examined this problem in 44,118 Portuguese–English response pairs from 14,706 survey-specific respondent records across four workplace surveys. One third of the pairs addressed mental health or well-being. Our analysis kept duplicate-aware retrieval, COMETKiwi quality estimates, language identification, and affective-lexicon coverage separate because they answer different questions. The retrieval encoder found the paired target or an exact normalized duplicate at rank one for 71.80% of English queries and 73.37% of Portuguese queries overall. Response length changed the picture. Rank-one recovery ranged from 8.39–12.96% for one- or two-word answers to 98.45–99.08% for answers longer than 20 words. COMETKiwi scores did not rise steadily with length, and about one quarter of responses in either language contained no emotion term covered by the lexicons. A failed match therefore does not, by itself, identify a poor translation. For brief and repetitive survey text, reference-free audits should report how distinguishable the response is and how much of it the diagnostic resources cover.
Public digital-phenotyping datasets share names for behaviors and mental-health outcomes while measuring them through different devices, processing rules, and instruments. We audit StudentLife, the College Experience Study (CES), CrossCheck, and Depresjon across feature definitions, temporal aggregation, population and study design, and outcome labels. Eight CES–CrossCheck daily correspondences yield raw-unit Hellinger distances from 0.104 for time in vehicle to 0.517 for unlock duration. Sensitivity analyses change the apparent similarity of several features: mobility distance rises from 0.148 to 0.307 after soft-outlier removal. Labels span PHQ-9, PHQ-2/PHQ-4, single-item and composite EMA, and clinician-rated MADRS. The effect audit separates between-person summaries from within-person variation. Between-person phone-feature correlations are small and uncertain; within-person stillness associations are 0.082 in CES and 0.053 in CrossCheck, with different outcome definitions. Depresjon shows lower mean activity in clinical cases than controls (d = −0.547). These findings make feature definition, preprocessing, label content, and statistical estimand explicit parts of cross-dataset evaluation. We provide field-level crosswalks, aggregate results, and reconstruction scripts for multimodal mental-health research.
In this work, we conduct an analysis to examine the consistency of large language models (LLMs) with respect to their own generated responses in an emotionally-driven conversational context. Specifically, the text generated by LLM is framed as a query to the same model, and its responses are subsequently assessed. This is performed with three queries across two dimensions of extreme and moderate emotions. The three queries are, in particular, false claim queries that contain inherently wrong assumptions (false presuppositions) in increasing order of intensity. Two commercial models, Claude-3.5-haiku, GPT4o-mini, and a medium-sized model, Mistral-7B, are considered in the study. Our findings indicate that LLMs exhibit below-average performance and remain vulnerable to false beliefs embedded within queries. This susceptibility is especially pronounced for moderate emotional content. Furthermore, an extended attention-score-based analysis highlights a shift in models’ priority from evaluative to generative. The results raise important considerations for LLMs’ deployment in high-stakes, emotionally sensitive contexts.
A familiar paper translates an English screener, fits a classifier to the new labels, reports a strong F1, and claims to detect depression in another language community. Standard metrics can look fine while construct equivalence, instrument adaptation, and annotation ecology remain unshown. We call this the label-translation fallacy. Building on unified validity theory, measurement modeling, data statements, and cultural psychiatry, we propose a four-layer reporting protocol (construct, instrument, annotation ecology, and licensed claim) for authors and reviewers at venues such as MultiPsyche. We provide minimal evidence thresholds grounded in COSMIN and psychometric standards, concrete templates for L3 annotation reporting, and a decision tree for mapping evidence to claims. Three one-pass case studies show operationalizability. The contribution is integrative rather than experimental: we combine existing validity tools into an operational checklist so that multilingual mental health metrics are interpreted only as strongly as their constructs allow.
High-quality automated ADHD assessment has the potential to increase screening rates among individuals in clinical settings, but these methods are far from perfect. Binary classification methods can obscure differences among clinical populations, and computational ADHD studies often evaluate binary diagnostic prediction against neurotypical controls. Such comparisons can obscure which measurements remain useful when ADHD is distinguished from other psychiatric conditions. We address these gaps using the HYPERAKTIV clinical cohort and four measurements: the Conners Continuous Performance Test II (CPT-II), self-report questionnaires, heart-rate variability (HRV), and actigraphy. We examine three aspects of multimodal ADHD assessments: what information different modalities provide for binary ADHD classification, how well multimodal measurements distinguish the clinical groups, and how patterns in the features relate to clinical diagnosis in unsupervised analysis. CPT-II combined with self-report achieved the highest binary AUC (0.908), but was not reliably better than self-report alone under nested cross-validation. In the three-way task, the inattentive group was the hardest to identify. Clustering produced feature profiles that crossed diagnostic groups. These results suggest that adding more modalities does not necessarily improve classification and that feature-space structure does not always align with diagnostic groups.
We present an initial study of robot-mediated multimodal mental-health screening for Chinese adolescents aged 12–15. MAPS (Multimodal Adolescent Psychological Screening) combines facial, acoustic, textual, and remotely estimated physiological signals collected through a standardized protocol of questionnaires, emotionally evocative tasks, and question answering. From 3,783 participants, 968 records were retained after consent, completeness, and quality-control filtering. We introduce GAME (Generalized Model with Attention and Multimodal EmbraceNet), a fusion model combining eight unimodal representations with relation-graph and attention features. Separate models are trained for 12 questionnaire- and expert-derived screening targets. Stratified 10-fold cross-validation yields 73.04–92.77% accuracy and 71.32–91.06% weighted F1. Feature ablations and descriptive contribution analyses suggest that representation use varies across targets. This initial study connects a portable interaction protocol with task-specific multimodal modeling in a Chinese school setting and provides a basis for further research on screening-label prediction. Future work will expand data collection, refine the fusion and training procedures, and evaluate generalization and screening utility through external and clinically supervised validation.
Large language models (LLMs) could widen access to mental health support, but support should also suit the person’s culture. This work asks what LLMs offer in the silver-linings exercise of Positive Affect Treatment (PAT), where a person lists positive aspects of an event. Following appraisal theory, we treat each silver lining as a claim that the event serves a goal. We wrote 18 high-school events in four versions, set in the United States or Thailand and in English or Thai. Two widely used LLMs gave six silver linings for each version, and two LLM coders coded which goal each silver lining serves and who gains from it, across all 864 lists of silver linings. Growth, what the student learned or can now do, was the most frequent goal in every condition and for both models. Without the country named, the share of linings that benefit others or the group changed by under three percentage points with language, setting, or their interaction. Naming the country raised the setting contrast by under five points. These results suggest little sensitivity to cultural cues, so a system that uses these models to guide the exercise cannot assume its suggestions fit the person asking.
Language-based schizophrenia detection has focused predominantly on English, leaving Arabic underrepresented in mental health NLP. We introduce an Arabic schizophrenia social media dataset (228 users, 228 controls; Cohen’s κ=0.76) and evaluate cross-lingual and cross-genre transfer across five primary settings and four reversed direction counterparts using XLM-RoBERTa-base, mBERT, and MARBERT. In-domain Arabic supervision yields the strongest performance, with MARBERT reaching 0.7706 macro-F1. In contrast, no primary transfer condition satisfies our criterion for robust transfer (macro-F1 > 0.50 and consistent performance across all three encoders). Reversed direction experiments suggest that genre transfer is directional: social to clinical transfer partially succeeds in the Arabic-only setting, whereas clinical to social transfer collapses. Results from translated clinical data further indicate that translation-based evaluation can introduce artifacts specific to encoder and direction. Overall, the findings support the need for native target language and target genre resources for reliable multilingual mental health evaluation.
Patient simulators are gaining traction in mental health training by providing scalable exposure to complex and sensitive patient interactions. Simulating depressed patients is challenging, as safety constraints and high patient variability complicate simulations and underscore the need for simulators that capture diverse and realistic patient behaviors. However, existing evaluations heavily rely on LLM-judges with poorly specified prompts and do not assess behavioral diversity. We introduce PSI-Bench, an automatic evaluation framework that provides interpretable, clinically meaningful diagnostics of depression patient simulator behavior across turn-, dialogue-, and population-level dimensions. Using PSI-Bench, we benchmark seven LLMs across two simulator frameworks and find that simulators produce overly long, lexically diverse responses, show reduced variability, and move through therapeutic stages and toward positive valence too quickly. We also show that the simulation framework has a larger impact on fidelity than the model scale. Results from a human study demonstrate that our benchmark is strongly aligned with judgments of mental health professionals. Our work reveals key limitations of current depression patient simulators and provides an interpretable, extensible benchmark to guide future simulator design and evaluation.
We present a multi-resolution framework for analyzing longitudinal Arabic mental-health discourse drawn from condition-specific Arabic-language X (formerly Twitter) communities. We model each user’s posting history as a calendar-time semantic trajectory and apply the Discrete Wavelet Transform (DWT) along the temporal axis to separate semantic variation occurring over finer and broader time scales. To interpret these temporal components linguistically, we introduce semantic gain, which measures how strongly corpus-derived phrases become aligned with the reconstructed semantic trajectory when information from a given temporal scale is introduced. We evaluate the framework on longitudinal trajectories from BPD-related and bipolar-related X communities using independent temporal-change measures, chronological permutation tests, cross-scale phrase distinctiveness, and matched high- versus low- gain events. Finer-scale DWT components align most strongly with local adjacent semantic change, whereas broader-scale components show greater correspondence with distributional change measured by Maximum Mean Discrepancy (MMD). Our findings suggest that longitudinal mental-health discourse on X exhibits multiscale semantic structure, with different forms of linguistic change emerging over different temporal horizons. Combining wavelet decomposition with phrase-based semantic interpretation provides a principled way to characterize both the temporal scale and linguistic content of discourse change.
We present PsychEthicsEval, a shared task that evaluates the alignment of large language models (LLMs) with Australian professional ethics guidelines in psychology and psychiatry. Building on PsychEthicsBench, the task comprises two complementary tracks: Multiple-Choice Questions (MCQ) assessing explicit ethical knowledge and Open-Ended Questions (OEQ) assessing ethical behavior in free-form responses. MCQ systems are ranked by Partial Credit (PC), while OEQ ethical rates are reported separately under rule-classifier and LLM-as-a-judge protocols. Four teams submitted final systems, with three systems evaluated in each track. AI4Health achieves the highest MCQ PC score of 0.832 and leads the rule-classifier OEQ evaluation, whereas XJTLU-EthicsPrompt leads under the LLM judge. Diagnostic analysis identifies judgments about professional identity and qualifications as the main source of the score gap between these two systems under the LLM judge, while jurisdictional applicability is a major source of score loss for group11 under the rule-classifier protocol, showing that comparisons of systems in open-ended ethical evaluation depend on how professional ethical criteria are applied.
We introduce PsyEthosRAG, a frozen-model system for aligning the ethical alignment of large language models (LLMs) in mental-health contexts, combining prompting that adapts to the role of the person asking with domain-specific Retrieval-Augmented Generation (RAG) over professional ethics codes. A single frozen backbone, prompting over a task-specific backbone, answers both subtasks of the PsychEthicsEval shared task: ethical knowledge, through multiple-choice questions (MCQ) answered by an independent per-option judgement that is robust to items which do not state how many options are correct; and ethical behaviour, through open-ended questions (OEQ) answered by a prompt that adapts to the inquirer, and is grounded in retrieved guideline passages. Experiments on the Phase-1 development data, combining exact-match scoring with an LLM-judge evaluation, demonstrate the effectiveness of PsyEthosRAG in producing accurate answers on the knowledge subtask and, on the behaviour subtask, responses that decline improper requests while remaining substantively engaged rather than refusing. A cross-model comparison and a few-shot ablation further motivate our minimal, frozen configuration.
Large language models are increasingly used in mental health applications, where responses must satisfy not only general safety requirements but also context-dependent ethical, professional, and jurisdictional constraints. We present a training-free inference framework for the MultiPsyche-2026 PsychEthicsEval shared task. For multiple-choice questions (Task I), the system uses ethics-aware structured prompting with independent option assessment and constrained machine-readable outputs. For open-ended questions (Task II), we introduce an identity-grounded and permission-bounded retrieval-augmented generation pipeline over curated Australian mental-health ethics resources. On the official hidden MCQ test set consisting of 282 samples, our system achieved a Partial Credit score of 0.768 (95% CI: 0.718–0.814) and an Exact Match score of 0.762. On 2,512 OEQ responses, the system obtained an ethical pass rate of 0.796 under the “rule pre-screening + classifier” evaluation and 0.621 under the independent LLM-as-a-Judge evaluation. The difference between the two OEQ evaluation scores indicates that the measurement results of open-ended ethical responses are influenced by how ethical violations are operationalized and evaluated.
Open-ended mental health ethics questions require models to balance stakeholder interests, ethical constraints, and conflicting courses of action. We present RoleEthic, a fourteen-stage structured prompting framework for the Open-Ended Question (OEQ) track of the PsychEthicsEval shared task at MultiPsyche 2026. The stages organize reasoning around case facts, stakeholder relationships and perspectives, and the AI’s role; gather and classify ethical considerations, apply a bottom-line check, and prioritize concerns; and guide response drafting and review for ethical consistency, role consistency, unsupported high-impact claims, and risky details. The framework runs within a single Qwen3-30B-A3B model call without explicit role labels or external verification and returns only the final response. In the official Phase 2 evaluation, RoleEthic achieves ethical rates of 0.562 under the rule pre-filter plus ModernBERT classifier approach and 0.522 under the GPT-5-mini judge approach, placing third among three submissions under each approach. Output diagnostics identify verbosity and additional contact-like details, while three purposively selected cases illustrate unsupported jurisdictional claims and role inconsistency in final responses.
Large language models are increasingly being explored for mental health applications, making reliable evaluation of their ethical behavior particularly important. However, professional ethics in psychology and psychiatry are jurisdiction-dependent and often require selecting multiple applicable principles rather than identifying a single correct response. The PsychEthicsEval shared task addresses this challenge through principle-grounded evaluation based on Australian mental health ethics. We investigate whether a compact language model can perform effectively in this setting without task-specific training. Our approach uses the frozen Qwen3.5-4B model with 4-bit quantized, prompting-only inference, combined with explicit Australian-jurisdiction grounding, answer-cardinality instructions, and deterministic answer extraction. In the original run on the 100-item labeled Phase 1 set, P2 achieves 70.00% exact match (EM) and 84.50% partial credit (PC), with 100.00% EM/PC on single-answer questions compared with 38.78% EM and 68.37% PC on multiple-answer questions. On the official Phase 2 evaluation of 282 held-out items, it achieves 65.20% EM and 77.50% PC (reported 95% confidence interval for PC: 73.40-81.20). A post-submission prompt-sensitivity analysis on Phase 1 found that an alternative wording reached 86.00% EM and 92.00% PC, while an option-wise diagnostic prompt performed worse overall. These exploratory results provide a reproducible low-resource prompting-only baseline and highlight the challenge of complete answer-set identification; alternative prompts were not evaluated on Phase 2.