PsyEthosRAG: Prompting and Retrieval for Aligning LLMs with Mental Health Ethics

Nijia Han, Yu Zhang, Jia Meng, Shuihua Wang


Abstract
We introduce PsyEthosRAG, a frozen-model system for aligning the ethical alignment of large language models (LLMs) in mental-health contexts, combining prompting that adapts to the role of the person asking with domain-specific Retrieval-Augmented Generation (RAG) over professional ethics codes. A single frozen backbone, prompting over a task-specific backbone, answers both subtasks of the PsychEthicsEval shared task: ethical knowledge, through multiple-choice questions (MCQ) answered by an independent per-option judgement that is robust to items which do not state how many options are correct; and ethical behaviour, through open-ended questions (OEQ) answered by a prompt that adapts to the inquirer, and is grounded in retrieved guideline passages. Experiments on the Phase-1 development data, combining exact-match scoring with an LLM-judge evaluation, demonstrate the effectiveness of PsyEthosRAG in producing accurate answers on the knowledge subtask and, on the behaviour subtask, responses that decline improper requests while remaining substantively engaged rather than refusing. A cross-model comparison and a few-shot ablation further motivate our minimal, frozen configuration.
Anthology ID:
2026.multipsyche-1.13
Volume:
Proceedings of the 1st Workshop on Multimodal, Multilingual, and Multicultural Mental Health and Psychotherapy (MultiPsyche)
Month:
November
Year:
2026
Address:
Hengqin, Zhuhai, China
Editors:
Zimu Wang, Yiwen Jiang, Yaling Shen, Xiangyu Zhao, Stephanie Fong, Xin Sun, Yi Zhou, Shaoxiong Ji, Robert Desimone, Barnaby Nelson, Dominic Dwyer, Zongyuan Ge
Venues:
MultiPsyche | WS
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
164–172
Language:
URL:
https://aclanthology.org/2026.multipsyche-1.13/
DOI:
Bibkey:
Cite (ACL):
Nijia Han, Yu Zhang, Jia Meng, and Shuihua Wang. 2026. PsyEthosRAG: Prompting and Retrieval for Aligning LLMs with Mental Health Ethics. In Proceedings of the 1st Workshop on Multimodal, Multilingual, and Multicultural Mental Health and Psychotherapy (MultiPsyche), pages 164–172, Hengqin, Zhuhai, China. Association for Computational Linguistics.
Cite (Informal):
PsyEthosRAG: Prompting and Retrieval for Aligning LLMs with Mental Health Ethics (Han et al., MultiPsyche 2026)
Copy Citation:
PDF:
https://aclanthology.org/2026.multipsyche-1.13.pdf