We introduce PsyEthosRAG, a frozen-model system for aligning the ethical alignment of large language models (LLMs) in mental-health contexts, combining prompting that adapts to the role of the person asking with domain-specific Retrieval-Augmented Generation (RAG) over professional ethics codes. A single frozen backbone, prompting over a task-specific backbone, answers both subtasks of the PsychEthicsEval shared task: ethical knowledge, through multiple-choice questions (MCQ) answered by an independent per-option judgement that is robust to items which do not state how many options are correct; and ethical behaviour, through open-ended questions (OEQ) answered by a prompt that adapts to the inquirer, and is grounded in retrieved guideline passages. Experiments on the Phase-1 development data, combining exact-match scoring with an LLM-judge evaluation, demonstrate the effectiveness of PsyEthosRAG in producing accurate answers on the knowledge subtask and, on the behaviour subtask, responses that decline improper requests while remaining substantively engaged rather than refusing. A cross-model comparison and a few-shot ablation further motivate our minimal, frozen configuration.

The rapid proliferation of large language models (LLMs) in medicine highlights their potential to revolutionize research in Traditional Chinese Medicine (TCM). While these models have shown great promise in assisting TCM practitioners by answering herb-related questions, generating syndrome-differentiation reports, and recommending classical formulas, a persistent challenge that arises is the issue of hallucination, where LLMs might produce content that appears plausible yet inaccurate. This issue has received limited attention within the context of TCM research, leaving a significant gap in understanding how hallucination manifests within the unique theoretical frameworks and diagnostic principles. Motivated by this phenomenon, we present TCMPHal, the first dataset specifically curated for hallucination detection in TCM pharmacy, comprising 10,000 high-quality question-answer pairs with hallucination annotations. Our experimental results across diverse LLMs, under standard, knowledge-based, and search engine-augmented conditions, demonstrate the capabilities and limitations of these models. A notable observation is that, for thinking LLMs, incorporating search engine results yields minimal improvement over their intrinsic reasoning abilities. We further conduct an in-depth error analysis, paving the way for future research directions in this domain. We release the TCMPHal dataset at https://github.com/hanninaa/TCMP.