Subhrajyoti Basu

Author directory

2026

Large language models are increasingly being explored for mental health applications, making reliable evaluation of their ethical behavior particularly important. However, professional ethics in psychology and psychiatry are jurisdiction-dependent and often require selecting multiple applicable principles rather than identifying a single correct response. The PsychEthicsEval shared task addresses this challenge through principle-grounded evaluation based on Australian mental health ethics. We investigate whether a compact language model can perform effectively in this setting without task-specific training. Our approach uses the frozen Qwen3.5-4B model with 4-bit quantized, prompting-only inference, combined with explicit Australian-jurisdiction grounding, answer-cardinality instructions, and deterministic answer extraction. In the original run on the 100-item labeled Phase 1 set, P2 achieves 70.00% exact match (EM) and 84.50% partial credit (PC), with 100.00% EM/PC on single-answer questions compared with 38.78% EM and 68.37% PC on multiple-answer questions. On the official Phase 2 evaluation of 282 held-out items, it achieves 65.20% EM and 77.50% PC (reported 95% confidence interval for PC: 73.40-81.20). A post-submission prompt-sensitivity analysis on Phase 1 found that an alternative wording reached 86.00% EM and 92.00% PC, while an option-wise diagnostic prompt performed worse overall. These exploratory results provide a reproducible low-resource prompting-only baseline and highlight the challenge of complete answer-set identification; alternative prompts were not evaluated on Phase 2.
We describe our system for the PsyDefDetect shared task at BioNLP 2026, which focuses onclassifying help-seeker utterances in multi-turn supportive conversations into nine psychological defense mechanism levels defined by the Defense Mechanism Rating Scales (DMRS). Our approach fine-tunes roberta-base using a composite training objective that combines focal loss, label smoothing, and squareroot dampened class weights to address the severe label imbalance present in the PSYDEFCONV corpus, where the dominant class constitutes 52% of the training data. The inputrepresentation is constructed by concatenating up to eight dialogue turns with role-specific tags, separated using RoBERTa’s native /s tokens, followed by the target utterance marked using a [TARGET] token. Model selection is performed using macro-F1 based early stopping on a stratified 15% validation split, along with cosine learning rate decay for stable optimization. Our best submission achieves an official Leaderboard 1 (positive classes) macroF1 score of 0.2556, ranking 11th among 21 registered teams.