Yiwen Jiang
Author directory2026
Overview of the PsychEthicsEval Shared Task at MultiPsyche 2026: Evaluating LLMs Against Australian Mental Health Ethics
Yaling Shen | Zimu Wang | Stephanie Fong | Yiwen Jiang | Feilong Tang | Qingyang Xu | Xiangyu Zhao | Zhongxing Xu | Jiahe Liu | Jinpeng Hu | Zongyuan Ge | Dominic Dwyer
Proceedings of the 1st Workshop on Multimodal, Multilingual, and Multicultural Mental Health and Psychotherapy (MultiPsyche)
Yaling Shen | Zimu Wang | Stephanie Fong | Yiwen Jiang | Feilong Tang | Qingyang Xu | Xiangyu Zhao | Zhongxing Xu | Jiahe Liu | Jinpeng Hu | Zongyuan Ge | Dominic Dwyer
Proceedings of the 1st Workshop on Multimodal, Multilingual, and Multicultural Mental Health and Psychotherapy (MultiPsyche)
We present PsychEthicsEval, a shared task that evaluates the alignment of large language models (LLMs) with Australian professional ethics guidelines in psychology and psychiatry. Building on PsychEthicsBench, the task comprises two complementary tracks: Multiple-Choice Questions (MCQ) assessing explicit ethical knowledge and Open-Ended Questions (OEQ) assessing ethical behavior in free-form responses. MCQ systems are ranked by Partial Credit (PC), while OEQ ethical rates are reported separately under rule-classifier and LLM-as-a-judge protocols. Four teams submitted final systems, with three systems evaluated in each track. AI4Health achieves the highest MCQ PC score of 0.832 and leads the rule-classifier OEQ evaluation, whereas XJTLU-EthicsPrompt leads under the LLM judge. Diagnostic analysis identifies judgments about professional identity and qualifications as the main source of the score gap between these two systems under the LLM judge, while jurisdictional applicability is a major source of score loss for group11 under the rule-classifier protocol, showing that comparisons of systems in open-ended ethical evaluation depend on how professional ethical criteria are applied.
Proceedings of the 1st Workshop on Multimodal, Multilingual, and Multicultural Mental Health and Psychotherapy (MultiPsyche)
Zimu Wang | Yiwen Jiang | Yaling Shen | Xiangyu Zhao | Stephanie Fong | Xin Sun | Yi Zhou | Shaoxiong Ji | Robert Desimone | Barnaby Nelson | Dominic Dwyer | Zongyuan Ge
Proceedings of the 1st Workshop on Multimodal, Multilingual, and Multicultural Mental Health and Psychotherapy (MultiPsyche)
Zimu Wang | Yiwen Jiang | Yaling Shen | Xiangyu Zhao | Stephanie Fong | Xin Sun | Yi Zhou | Shaoxiong Ji | Robert Desimone | Barnaby Nelson | Dominic Dwyer | Zongyuan Ge
Proceedings of the 1st Workshop on Multimodal, Multilingual, and Multicultural Mental Health and Psychotherapy (MultiPsyche)
Do No Harm: Exposing Hidden Vulnerabilities of LLMs via Persona-based Client Simulation Attack in Psychological Counseling
Qingyang Xu | Yaling Shen | Stephanie Fong | Zimu Wang | Yiwen Jiang | Xiangyu Zhao | Jiahe Liu | Zhongxing Xu | Vincent Lee | Zongyuan Ge
Findings of the Association for Computational Linguistics: EMNLP 2026
Qingyang Xu | Yaling Shen | Stephanie Fong | Zimu Wang | Yiwen Jiang | Xiangyu Zhao | Jiahe Liu | Zhongxing Xu | Vincent Lee | Zongyuan Ge
Findings of the Association for Computational Linguistics: EMNLP 2026
The increasing use of large language models (LLMs) in mental healthcare raises safety concerns in high-stakes therapeutic interactions. A key challenge is distinguishing therapeutic empathy from maladaptive validation, where supportive responses may inadvertently reinforce harmful beliefs or behaviors in multi-turn conversations. This risk is largely overlooked by existing red-teaming frameworks, which focus mainly on generic harms or optimization-based attacks. To address this gap, we introduce Persona-based Client Simulation Attack (PCSA), the first red-teaming framework that simulates clients in psychological counseling through coherent, persona-driven client dialogues to expose vulnerabilities in psychological safety alignment. Experiments on eight general and mental health-specialized LLMs show that PCSA substantially outperforms four competitive baselines. Perplexity analysis and human inspection further indicate that PCSA generates more natural and realistic dialogues. Our results reveal that current LLMs remain vulnerable to domain-specific adversarial tactics, providing unauthorized medical advice, reinforcing delusions, and implicitly encouraging risky actions.
AudioProcessBench: Benchmark for Identifying Process Errors in Audio-Grounded Reasoning
Xiangyu Zhao | Junyu Yan | Yaling Shen | Stephanie Fong | Zimu Wang | Yiwen Jiang | Qingyang Xu | Jiahe Liu | Dominic Dwyer | Zongyuan Ge
Findings of the Association for Computational Linguistics: EMNLP 2026
Xiangyu Zhao | Junyu Yan | Yaling Shen | Stephanie Fong | Zimu Wang | Yiwen Jiang | Qingyang Xu | Jiahe Liu | Dominic Dwyer | Zongyuan Ge
Findings of the Association for Computational Linguistics: EMNLP 2026
Large audio-language models (LALMs) increasingly use explicit reasoning traces for complex audio understanding, yet the evaluation of reasoning quality remains underexplored. Although process-level benchmarks for process reward models (PRMs) have advanced reasoning evaluation in text and multi-modal domains, comparable evaluation for audio reasoning remains limited. In this paper, we present AudioProcessBench, a comprehensive benchmark for step-level process error identification in audio reasoning. AudioProcessBench contains diverse reasoning traces generated by 6 audio and omni language models. Each trace is segmented into discrete reasoning steps and annotated with binary step correctness and fine-grained error types. Our benchmark evaluates models under three complementary paradigms: (1) step correctness identification, (2) error-type-conditioned detection for diagnosing audio-specific verifier capacities, and (3) chain-level aggregation, where verifiers select or aggregate among multiple reasoning traces for the same question. This design enables a systematic analysis of whether current models can detect process errors, whether their weaknesses differ across audio-specific error types, and whether process verification translates into improved answer selection. AudioProcessBench provides a testbed for future research on audio reasoning verifiers, process reward models, and reliable omni-modal reasoning.
VIBE-Bench: Evaluating Personalized Large Language Models When Profiles Don’t Mean Preferences
Yiwen Jiang | Yang Deng | Stephanie Fong | Zimu Wang | Yaling Shen | Wei Feng | Hongxi Yang | Xiangyu Zhao | Zhongxing Xu | Deval Mehta | Xuelian Cheng | Zongyuan Ge
Findings of the Association for Computational Linguistics: EMNLP 2026
Yiwen Jiang | Yang Deng | Stephanie Fong | Zimu Wang | Yaling Shen | Wei Feng | Hongxi Yang | Xiangyu Zhao | Zhongxing Xu | Deval Mehta | Xuelian Cheng | Zongyuan Ge
Findings of the Association for Computational Linguistics: EMNLP 2026
Personalized Large Language Models (PLLMs) aim to tailor responses to individual users, where a central challenge is preference reasoning: inferring query-relevant preferences from user-related history. Existing benchmarks, however, largely assume that such preference can be retrieved from semantically related history. We study an underexplored but practically important regime, profile-preference conceptual misalignment (PRCM), where observable profile cues and query-specific preferences lie in different concept spaces, making semantic retrieval inconsistent for personalization. We introduce Vibe-Bench, a benchmark with two psychology-grounded tasks, 3,504 personas and 12,239 dialogues, including a manually verified gold test set, and requires cross-concept preference reasoning beyond surface semantic overlap. Experiments with several personalization methods show that current PLLMs largely rely on shallow semantic correlations and fail to acquire robust cross-concept mappings. These findings establish PRCM as a distinct failure regime in PLLMs and position Vibe-Bench as a focused testbed for advancing preference reasoning beyond semantic matching.
PsychEthicsBench: Evaluating Large Language Models Against Australian Mental Health Ethics
Yaling Shen | Stephanie Fong | Yiwen Jiang | Zimu Wang | Feilong Tang | Qingyang Xu | Xiangyu Zhao | Zhongxing Xu | Jiahe Liu | Jinpeng Hu | Dominic Dwyer | Zongyuan Ge
Findings of the Association for Computational Linguistics: ACL 2026
Yaling Shen | Stephanie Fong | Yiwen Jiang | Zimu Wang | Feilong Tang | Qingyang Xu | Xiangyu Zhao | Zhongxing Xu | Jiahe Liu | Jinpeng Hu | Dominic Dwyer | Zongyuan Ge
Findings of the Association for Computational Linguistics: ACL 2026
The increasing integration of large language models (LLMs) into mental health applications necessitates robust frameworks for evaluating professional safety alignment. Current evaluative approaches primarily rely on refusal-based safety signals, which offer limited insight into the nuanced behaviors required in clinical practice. In mental health, clinically inadequate refusals can be perceived as unempathetic and discourage help-seeking. To address this gap, we move beyond refusal-centric metrics and introduce PsychEthicsBench, the first principle-grounded benchmark based on Australian psychology and psychiatry guidelines, designed to evaluate LLMs’ ethical knowledge and behavioral responses through multiple-choice and open-ended tasks with fine-grained ethicality annotations. Empirical results across 14 models reveal that refusal rates are poor indicators of ethical behavior, revealing a significant divergence between safety triggers and clinical appropriateness. Notably, we find that domain-specific fine-tuning can degrade ethical robustness, as several specialized models underperform their base backbones in ethical alignment. PsychEthicsBench provides a foundation for systematic, jurisdiction-aware evaluation of LLMs in mental health, encouraging more responsible development in this domain.
It Hears, It Sees too: Multi-Modal LLM for Depression Detection By Integrating Visual Understanding into Audio Language Models
Xiangyu Zhao | Yaling Shen | Yiwen Jiang | Zimu Wang | Jiahe Liu | Maxmartwell H Cheng | Guilherme C Oliveira | Robert Desimone | Dominic Dwyer | Zongyuan Ge
Findings of the Association for Computational Linguistics: IJCNLP-AACL 2026
Xiangyu Zhao | Yaling Shen | Yiwen Jiang | Zimu Wang | Jiahe Liu | Maxmartwell H Cheng | Guilherme C Oliveira | Robert Desimone | Dominic Dwyer | Zongyuan Ge
Findings of the Association for Computational Linguistics: IJCNLP-AACL 2026
Depression is one of the most prevalent mental health disorders globally. In recent years, multi-modal data, such as speech, video, and transcripts, has been increasingly used to develop AI-assisted depression assessment systems. Large language models (LLMs) have further advanced this field due to their strong language understanding and generalization capabilities. However, conventional LLMs remain text-centric and cannot process the rich non-verbal cues found in audio and visual modalities, which are critical components in mental health evaluation. While multi-modal LLMs offer a promising direction, few are tailored for psychological applications. In this study, we propose a novel multi-modal LLM framework for depression detection. Our approach augments an audio language model with visual understanding and aligns audio-visual features at the timestamp level. This fine-grained alignment improves modeling of temporal dynamics across modalities while reducing the need for extensive training data and computational resources. Experiments on the DAIC-WoZ and E-DAIC datasets demonstrate that our model consistently outperforms both single-modality approaches and previous multi-modal methods. Moreover, the proposed framework can be extended to incorporate additional physiological signals, paving the way for broader clinical applications beyond mental health.
Steering the Compass: Aligning Dynamic Psychological Counseling Conversations with Cognitive Behavioral Therapy Strategies
Zimu Wang | Yiwen Jiang | Xiangyu Zhao | Yaling Shen | Jiahe Liu | Stephanie Fong | Maxmartwell H Cheng | Guilherme C Oliveira | Anh Nguyen | Robert Desimone | Barnaby Nelson | Dominic Dwyer | Zongyuan Ge
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Zimu Wang | Yiwen Jiang | Xiangyu Zhao | Yaling Shen | Jiahe Liu | Stephanie Fong | Maxmartwell H Cheng | Guilherme C Oliveira | Anh Nguyen | Robert Desimone | Barnaby Nelson | Dominic Dwyer | Zongyuan Ge
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Recent advancements in large language models have revolutionized the field of psychological counseling, especially in the context of Cognitive Behavioral Therapy (CBT). While the success of CBT relies heavily on dynamic decision-making informed by the client’s real-time mental state, this aspect has often been overlooked in current research, limiting both flexibility and therapeutic outcomes. In this paper, we introduce StratCBT, a dataset specifically designed for psychological counseling conversations with CBT Strategies, consisting of 9,688 sessions and around 256K utterances, with each counselor’s response aligned with one of eight distinct strategies. The creation of StratCBT involves modeling clients based on their negative thoughts and generating high-quality counseling conversations through self-chat, incorporating realistic sessions as guidance, thereby significantly surpassing existing datasets in both general counseling and CBT-specific skills. We conduct extensive experiments to demonstrate the effectiveness of strategy-aligned generation and evaluate its efficacy in delivering professional and effective counseling with LLM-simulated clients to reflect real-world scenarios. The dataset can be obtained from https://github.com/zimuwangnlp/StratCBT.
SDARE-Bench: Evaluating Large Language Models on Conversational Stigma Detection and Response in Dyadic and Group Dialogue
Stephanie Fong | Yiwen Jiang | Zimu Wang | Hongxi Yang | Yaling Shen | Hiu Weh Naomi Chow | Heung Ying Lai | Xiangyu Zhao | Qingyang Xu | Zhongxing Xu | Jiahe Liu | Guilherme C Oliveira | Vincent Lee | Zongyuan Ge | Dominic Dwyer
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Stephanie Fong | Yiwen Jiang | Zimu Wang | Hongxi Yang | Yaling Shen | Hiu Weh Naomi Chow | Heung Ying Lai | Xiangyu Zhao | Qingyang Xu | Zhongxing Xu | Jiahe Liu | Guilherme C Oliveira | Vincent Lee | Zongyuan Ge | Dominic Dwyer
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Large Language Models (LLMs) are increasingly used in advice seeking and decision making that may affect social judgements. Despite stigma’s profound effects on people and communities, benchmarks remain scarce. Existing general-domain evaluations typically rely on static prompts and fixed-format tasks, overlooking conversational contexts and audience effects in everyday communication. To address these gaps, we introduce SDARE-Bench, the first scenario-based benchmark evaluating both stigma detection and open-ended response generation in LLMs, comprising 1,138 dyadic queries and 1,388 group dialogues. Empirical results across 8 LLMs consistently demonstrate poor identification of stigma components, especially in group dialogues. In open-ended response generation, stigma expression was substantially higher in group settings than in dyadic, with weaker resistance to stigma and more unrealistic advice. Responses were evaluated using a classifier trained on 1,392 human annotated responses. In constructed group pressure settings, stigma expression rates further increased to a striking average of 97.5%. Our findings identify stigma response as a recurring LLM safety vulnerability, especially in socially complex conversational contexts.
CHiRPE: A Step Towards Real-World Clinical NLP with Clinician-Oriented Model Explanations
Stephanie Fong | Zimu Wang | Guilherme C Oliveira | Xiangyu Zhao | Yiwen Jiang | Jiahe Liu | Beau-Luke Colton | Scott W. Woods | Martha Shenton | Barnaby Nelson | Zongyuan Ge | Dominic Dwyer
Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 2: Short Papers)
Stephanie Fong | Zimu Wang | Guilherme C Oliveira | Xiangyu Zhao | Yiwen Jiang | Jiahe Liu | Beau-Luke Colton | Scott W. Woods | Martha Shenton | Barnaby Nelson | Zongyuan Ge | Dominic Dwyer
Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 2: Short Papers)
The medical adoption of NLP tools requires interpretability by end users, yet traditional explainable AI (XAI) methods are misaligned with clinical reasoning and lack clinician input. We introduce CHiRPE (Clinical High-Risk Prediction with Explainability), an NLP pipeline that takes transcribed semi-structured clinical interviews to: (i) predict psychosis risk; and (ii) generate novel SHAP explanation formats co-developed with clinicians. Trained on 944 semi-structured interview transcripts across 24 international clinics of the AMP-SCZ study, the CHiRPE pipeline integrates symptom-domain mapping, LLM summarisation, and BERT classification. CHiRPE achieved over 90% accuracy across three BERT variants and outperformed baseline models. Explanation formats were evaluated by 28 clinical experts who indicated a strong preference for our novel concept-guided explanations, especially hybrid graph-and-text summary formats. CHiRPE demonstrates that clinically-guided model development produces both accurate and interpretable results. Our next step is focused on real-world testing across our 24 international sites.
2025
WISE: Weak-Supervision-Guided Step-by-Step Explanations for Multimodal LLMs in Image Classification
Yiwen Jiang | Deval Mehta | Siyuan Yan | Yaling Shen | Zimu Wang | Zongyuan Ge
Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing
Yiwen Jiang | Deval Mehta | Siyuan Yan | Yaling Shen | Zimu Wang | Zongyuan Ge
Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing
Multimodal Large Language Models (MLLMs) have shown promise in visual-textual reasoning, with Multimodal Chain-of-Thought (MCoT) prompting significantly enhancing interpretability. However, existing MCoT methods rely on rationale-rich datasets and largely focus on inter-object reasoning, overlooking the intra-object understanding crucial for image classification. To address this gap, we propose WISE, a Weak-supervision-guided Step-by-step Explanation method that augments any image classification dataset with MCoTs by reformulating the concept-based representations from Concept Bottleneck Models (CBMs) into concise, interpretable reasoning chains under weak supervision. Experiments across ten datasets show that our generated MCoTs not only improve interpretability by 37% but also lead to gains in classification accuracy when used to fine-tune MLLMs. Our work bridges concept-based interpretability and generative MCoT reasoning, providing a generalizable framework for enhancing MLLMs in fine-grained visual understanding.
Enhancing Interpretable Image Classification Through LLM Agents and Conditional Concept Bottleneck Models
Yiwen Jiang | Deval Mehta | Wei Feng | Zongyuan Ge
Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
Yiwen Jiang | Deval Mehta | Wei Feng | Zongyuan Ge
Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
Concept Bottleneck Models (CBMs) decompose image classification into a process governed by interpretable, human-readable concepts. Recent advances in CBMs have used Large Language Models (LLMs) to generate candidate concepts. However, a critical question remains: What is the optimal number of concepts to use? Current concept banks suffer from redundancy or insufficient coverage. To address this issue, we introduce a dynamic, agent-based approach that adjusts the concept bank in response to environmental feedback, optimizing the number of concepts for sufficiency yet concise coverage. Moreover, we propose Conditional Concept Bottleneck Models (CoCoBMs) to overcome the limitations in traditional CBMs’ concept scoring mechanisms. It enhances the accuracy of assessing each concept’s contribution to classification tasks and feature an editable matrix that allows LLMs to correct concept scores that conflict with their internal knowledge. Our evaluations across 6 datasets show that our method not only improves classification accuracy by 6% but also enhances interpretability assessments by 30%.
Search
Co-authors
- Zongyuan Ge 12
- Zimu Wang 11
- Yaling Shen 10
- Stephanie Fong 9
- Dominic Dwyer 8
- Xiangyu Zhao 8
- Jiahe Liu 7
- Qingyang Xu 5
- Zhongxing Xu 5
- Guilherme C Oliveira 4
- Robert Desimone 3
- Deval Mehta 3
- Barnaby Nelson 3
- Maxmartwell H Cheng 2
- Wei Feng 2
- Jinpeng Hu 2
- Vincent Lee 2
- Feilong Tang 2
- Hongxi Yang 2
- Xiangyu Zhao 2
- Xuelian Cheng 1
- Hiu Weh Naomi Chow 1
- Beau-Luke Colton 1
- Yang Deng 1
- Shaoxiong Ji 1
- Heung Ying Lai 1
- Jiahe Liu 1
- Anh Nguyen 1
- Martha Shenton 1
- Xin Sun 1
- Scott W. Woods 1
- Junyu Yan 1
- Siyuan Yan 1
- Yi Zhou 1