Dominic Dwyer
Author directory2026
Overview of the PsychEthicsEval Shared Task at MultiPsyche 2026: Evaluating LLMs Against Australian Mental Health Ethics
Yaling Shen | Zimu Wang | Stephanie Fong | Yiwen Jiang | Feilong Tang | Qingyang Xu | Xiangyu Zhao | Zhongxing Xu | Jiahe Liu | Jinpeng Hu | Zongyuan Ge | Dominic Dwyer
Proceedings of the 1st Workshop on Multimodal, Multilingual, and Multicultural Mental Health and Psychotherapy (MultiPsyche)
Yaling Shen | Zimu Wang | Stephanie Fong | Yiwen Jiang | Feilong Tang | Qingyang Xu | Xiangyu Zhao | Zhongxing Xu | Jiahe Liu | Jinpeng Hu | Zongyuan Ge | Dominic Dwyer
Proceedings of the 1st Workshop on Multimodal, Multilingual, and Multicultural Mental Health and Psychotherapy (MultiPsyche)
We present PsychEthicsEval, a shared task that evaluates the alignment of large language models (LLMs) with Australian professional ethics guidelines in psychology and psychiatry. Building on PsychEthicsBench, the task comprises two complementary tracks: Multiple-Choice Questions (MCQ) assessing explicit ethical knowledge and Open-Ended Questions (OEQ) assessing ethical behavior in free-form responses. MCQ systems are ranked by Partial Credit (PC), while OEQ ethical rates are reported separately under rule-classifier and LLM-as-a-judge protocols. Four teams submitted final systems, with three systems evaluated in each track. AI4Health achieves the highest MCQ PC score of 0.832 and leads the rule-classifier OEQ evaluation, whereas XJTLU-EthicsPrompt leads under the LLM judge. Diagnostic analysis identifies judgments about professional identity and qualifications as the main source of the score gap between these two systems under the LLM judge, while jurisdictional applicability is a major source of score loss for group11 under the rule-classifier protocol, showing that comparisons of systems in open-ended ethical evaluation depend on how professional ethical criteria are applied.
Proceedings of the 1st Workshop on Multimodal, Multilingual, and Multicultural Mental Health and Psychotherapy (MultiPsyche)
Zimu Wang | Yiwen Jiang | Yaling Shen | Xiangyu Zhao | Stephanie Fong | Xin Sun | Yi Zhou | Shaoxiong Ji | Robert Desimone | Barnaby Nelson | Dominic Dwyer | Zongyuan Ge
Proceedings of the 1st Workshop on Multimodal, Multilingual, and Multicultural Mental Health and Psychotherapy (MultiPsyche)
Zimu Wang | Yiwen Jiang | Yaling Shen | Xiangyu Zhao | Stephanie Fong | Xin Sun | Yi Zhou | Shaoxiong Ji | Robert Desimone | Barnaby Nelson | Dominic Dwyer | Zongyuan Ge
Proceedings of the 1st Workshop on Multimodal, Multilingual, and Multicultural Mental Health and Psychotherapy (MultiPsyche)
AudioProcessBench: Benchmark for Identifying Process Errors in Audio-Grounded Reasoning
Xiangyu Zhao | Junyu Yan | Yaling Shen | Stephanie Fong | Zimu Wang | Yiwen Jiang | Qingyang Xu | Jiahe Liu | Dominic Dwyer | Zongyuan Ge
Findings of the Association for Computational Linguistics: EMNLP 2026
Xiangyu Zhao | Junyu Yan | Yaling Shen | Stephanie Fong | Zimu Wang | Yiwen Jiang | Qingyang Xu | Jiahe Liu | Dominic Dwyer | Zongyuan Ge
Findings of the Association for Computational Linguistics: EMNLP 2026
Large audio-language models (LALMs) increasingly use explicit reasoning traces for complex audio understanding, yet the evaluation of reasoning quality remains underexplored. Although process-level benchmarks for process reward models (PRMs) have advanced reasoning evaluation in text and multi-modal domains, comparable evaluation for audio reasoning remains limited. In this paper, we present AudioProcessBench, a comprehensive benchmark for step-level process error identification in audio reasoning. AudioProcessBench contains diverse reasoning traces generated by 6 audio and omni language models. Each trace is segmented into discrete reasoning steps and annotated with binary step correctness and fine-grained error types. Our benchmark evaluates models under three complementary paradigms: (1) step correctness identification, (2) error-type-conditioned detection for diagnosing audio-specific verifier capacities, and (3) chain-level aggregation, where verifiers select or aggregate among multiple reasoning traces for the same question. This design enables a systematic analysis of whether current models can detect process errors, whether their weaknesses differ across audio-specific error types, and whether process verification translates into improved answer selection. AudioProcessBench provides a testbed for future research on audio reasoning verifiers, process reward models, and reliable omni-modal reasoning.
PsychEthicsBench: Evaluating Large Language Models Against Australian Mental Health Ethics
Yaling Shen | Stephanie Fong | Yiwen Jiang | Zimu Wang | Feilong Tang | Qingyang Xu | Xiangyu Zhao | Zhongxing Xu | Jiahe Liu | Jinpeng Hu | Dominic Dwyer | Zongyuan Ge
Findings of the Association for Computational Linguistics: ACL 2026
Yaling Shen | Stephanie Fong | Yiwen Jiang | Zimu Wang | Feilong Tang | Qingyang Xu | Xiangyu Zhao | Zhongxing Xu | Jiahe Liu | Jinpeng Hu | Dominic Dwyer | Zongyuan Ge
Findings of the Association for Computational Linguistics: ACL 2026
The increasing integration of large language models (LLMs) into mental health applications necessitates robust frameworks for evaluating professional safety alignment. Current evaluative approaches primarily rely on refusal-based safety signals, which offer limited insight into the nuanced behaviors required in clinical practice. In mental health, clinically inadequate refusals can be perceived as unempathetic and discourage help-seeking. To address this gap, we move beyond refusal-centric metrics and introduce PsychEthicsBench, the first principle-grounded benchmark based on Australian psychology and psychiatry guidelines, designed to evaluate LLMs’ ethical knowledge and behavioral responses through multiple-choice and open-ended tasks with fine-grained ethicality annotations. Empirical results across 14 models reveal that refusal rates are poor indicators of ethical behavior, revealing a significant divergence between safety triggers and clinical appropriateness. Notably, we find that domain-specific fine-tuning can degrade ethical robustness, as several specialized models underperform their base backbones in ethical alignment. PsychEthicsBench provides a foundation for systematic, jurisdiction-aware evaluation of LLMs in mental health, encouraging more responsible development in this domain.
It Hears, It Sees too: Multi-Modal LLM for Depression Detection By Integrating Visual Understanding into Audio Language Models
Xiangyu Zhao | Yaling Shen | Yiwen Jiang | Zimu Wang | Jiahe Liu | Maxmartwell H Cheng | Guilherme C Oliveira | Robert Desimone | Dominic Dwyer | Zongyuan Ge
Findings of the Association for Computational Linguistics: IJCNLP-AACL 2026
Xiangyu Zhao | Yaling Shen | Yiwen Jiang | Zimu Wang | Jiahe Liu | Maxmartwell H Cheng | Guilherme C Oliveira | Robert Desimone | Dominic Dwyer | Zongyuan Ge
Findings of the Association for Computational Linguistics: IJCNLP-AACL 2026
Depression is one of the most prevalent mental health disorders globally. In recent years, multi-modal data, such as speech, video, and transcripts, has been increasingly used to develop AI-assisted depression assessment systems. Large language models (LLMs) have further advanced this field due to their strong language understanding and generalization capabilities. However, conventional LLMs remain text-centric and cannot process the rich non-verbal cues found in audio and visual modalities, which are critical components in mental health evaluation. While multi-modal LLMs offer a promising direction, few are tailored for psychological applications. In this study, we propose a novel multi-modal LLM framework for depression detection. Our approach augments an audio language model with visual understanding and aligns audio-visual features at the timestamp level. This fine-grained alignment improves modeling of temporal dynamics across modalities while reducing the need for extensive training data and computational resources. Experiments on the DAIC-WoZ and E-DAIC datasets demonstrate that our model consistently outperforms both single-modality approaches and previous multi-modal methods. Moreover, the proposed framework can be extended to incorporate additional physiological signals, paving the way for broader clinical applications beyond mental health.
Steering the Compass: Aligning Dynamic Psychological Counseling Conversations with Cognitive Behavioral Therapy Strategies
Zimu Wang | Yiwen Jiang | Xiangyu Zhao | Yaling Shen | Jiahe Liu | Stephanie Fong | Maxmartwell H Cheng | Guilherme C Oliveira | Anh Nguyen | Robert Desimone | Barnaby Nelson | Dominic Dwyer | Zongyuan Ge
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Zimu Wang | Yiwen Jiang | Xiangyu Zhao | Yaling Shen | Jiahe Liu | Stephanie Fong | Maxmartwell H Cheng | Guilherme C Oliveira | Anh Nguyen | Robert Desimone | Barnaby Nelson | Dominic Dwyer | Zongyuan Ge
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Recent advancements in large language models have revolutionized the field of psychological counseling, especially in the context of Cognitive Behavioral Therapy (CBT). While the success of CBT relies heavily on dynamic decision-making informed by the client’s real-time mental state, this aspect has often been overlooked in current research, limiting both flexibility and therapeutic outcomes. In this paper, we introduce StratCBT, a dataset specifically designed for psychological counseling conversations with CBT Strategies, consisting of 9,688 sessions and around 256K utterances, with each counselor’s response aligned with one of eight distinct strategies. The creation of StratCBT involves modeling clients based on their negative thoughts and generating high-quality counseling conversations through self-chat, incorporating realistic sessions as guidance, thereby significantly surpassing existing datasets in both general counseling and CBT-specific skills. We conduct extensive experiments to demonstrate the effectiveness of strategy-aligned generation and evaluate its efficacy in delivering professional and effective counseling with LLM-simulated clients to reflect real-world scenarios. The dataset can be obtained from https://github.com/zimuwangnlp/StratCBT.
SDARE-Bench: Evaluating Large Language Models on Conversational Stigma Detection and Response in Dyadic and Group Dialogue
Stephanie Fong | Yiwen Jiang | Zimu Wang | Hongxi Yang | Yaling Shen | Hiu Weh Naomi Chow | Heung Ying Lai | Xiangyu Zhao | Qingyang Xu | Zhongxing Xu | Jiahe Liu | Guilherme C Oliveira | Vincent Lee | Zongyuan Ge | Dominic Dwyer
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Stephanie Fong | Yiwen Jiang | Zimu Wang | Hongxi Yang | Yaling Shen | Hiu Weh Naomi Chow | Heung Ying Lai | Xiangyu Zhao | Qingyang Xu | Zhongxing Xu | Jiahe Liu | Guilherme C Oliveira | Vincent Lee | Zongyuan Ge | Dominic Dwyer
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Large Language Models (LLMs) are increasingly used in advice seeking and decision making that may affect social judgements. Despite stigma’s profound effects on people and communities, benchmarks remain scarce. Existing general-domain evaluations typically rely on static prompts and fixed-format tasks, overlooking conversational contexts and audience effects in everyday communication. To address these gaps, we introduce SDARE-Bench, the first scenario-based benchmark evaluating both stigma detection and open-ended response generation in LLMs, comprising 1,138 dyadic queries and 1,388 group dialogues. Empirical results across 8 LLMs consistently demonstrate poor identification of stigma components, especially in group dialogues. In open-ended response generation, stigma expression was substantially higher in group settings than in dyadic, with weaker resistance to stigma and more unrealistic advice. Responses were evaluated using a classifier trained on 1,392 human annotated responses. In constructed group pressure settings, stigma expression rates further increased to a striking average of 97.5%. Our findings identify stigma response as a recurring LLM safety vulnerability, especially in socially complex conversational contexts.
CHiRPE: A Step Towards Real-World Clinical NLP with Clinician-Oriented Model Explanations
Stephanie Fong | Zimu Wang | Guilherme C Oliveira | Xiangyu Zhao | Yiwen Jiang | Jiahe Liu | Beau-Luke Colton | Scott W. Woods | Martha Shenton | Barnaby Nelson | Zongyuan Ge | Dominic Dwyer
Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 2: Short Papers)
Stephanie Fong | Zimu Wang | Guilherme C Oliveira | Xiangyu Zhao | Yiwen Jiang | Jiahe Liu | Beau-Luke Colton | Scott W. Woods | Martha Shenton | Barnaby Nelson | Zongyuan Ge | Dominic Dwyer
Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 2: Short Papers)
The medical adoption of NLP tools requires interpretability by end users, yet traditional explainable AI (XAI) methods are misaligned with clinical reasoning and lack clinician input. We introduce CHiRPE (Clinical High-Risk Prediction with Explainability), an NLP pipeline that takes transcribed semi-structured clinical interviews to: (i) predict psychosis risk; and (ii) generate novel SHAP explanation formats co-developed with clinicians. Trained on 944 semi-structured interview transcripts across 24 international clinics of the AMP-SCZ study, the CHiRPE pipeline integrates symptom-domain mapping, LLM summarisation, and BERT classification. CHiRPE achieved over 90% accuracy across three BERT variants and outperformed baseline models. Explanation formats were evaluated by 28 clinical experts who indicated a strong preference for our novel concept-guided explanations, especially hybrid graph-and-text summary formats. CHiRPE demonstrates that clinically-guided model development produces both accurate and interpretable results. Our next step is focused on real-world testing across our 24 international sites.
Search
Co-authors
- Zongyuan Ge 8
- Yiwen Jiang 8
- Zimu Wang 8
- Stephanie Fong 7
- Yaling Shen 7
- Jiahe Liu 6
- Xiangyu Zhao 6
- Guilherme C Oliveira 4
- Qingyang Xu 4
- Robert Desimone 3
- Barnaby Nelson 3
- Zhongxing Xu 3
- Maxmartwell H Cheng 2
- Jinpeng Hu 2
- Feilong Tang 2
- Xiangyu Zhao 2
- Hiu Weh Naomi Chow 1
- Beau-Luke Colton 1
- Shaoxiong Ji 1
- Heung Ying Lai 1
- Vincent Lee 1
- Jiahe Liu 1
- Anh Nguyen 1
- Martha Shenton 1
- Xin Sun 1
- Scott W. Woods 1
- Junyu Yan 1
- Hongxi Yang 1
- Yi Zhou 1