Stephen Swift
Author directory2026
Unseen Harm: Measuring Cross-Script Safety Inconsistency with "Missed-in-Urdu" Scores in LLM Hate Speech Detection.
Fawzia Kara-Isitt | Sonal Khosla | Stephen Swift
Proceedings of the 10th Workshop on Online Abuse and Harms (WOAH)
Fawzia Kara-Isitt | Sonal Khosla | Stephen Swift
Proceedings of the 10th Workshop on Online Abuse and Harms (WOAH)
This paper investigates the near-total absence of Urdu from mainstream LLM safety evaluation and nine years of WOAH proceedings. A complete enumeration of all 205 papers across nine ALW/WOAH editions confirms zero dedicated Urdu papers across the entire period. To assess whether this absence has measurable consequences, five large language models (GPT-4o, Claude Sonnet, Gemini 2.5 Flash, Llama-3.1, and Qwen-2.5) were evaluated across six datasets spanning Nastaliq Urdu, Roman Urdu, English, and code-switched Urdu-English, producing approximately 28,000 classifications. Label instability between original-script and English-translation classification ranged from 15.9% to 31.6%, and a missed-in-Urdu rate content flagged as harmful in English but passed as normal in Urdu ranged from 2.4% to 9.9%. Pairwise significance testing confirms that frontier closed models form a statistically indistinguishable cluster significantly outperforming smaller open-weight models on both metrics. The paper also identifies the absence of a dedicated Nastaliq Urdu with English code-switched hate speech dataset as a structural gap, and raises questions about annotation bias in existing Urdu corpora. The findings demonstrate that Urdu speakers are exposed to measurably weaker content moderation safety assurance, a gap that this paper documents, quantifies, and calls on the research community to address, considering that it is in the top 10 most spoken and used language in the world.
2025
From Evidence Mining to Meta-Prediction: a Gradient of Methodologies for Task-Specific Challenges in Psychological Assessment
Federico Ravenda | Fawzia-Zehra Kara-Isitt | Stephen Swift | Antonietta Mira | Andrea Raballo
Proceedings of the 10th Workshop on Computational Linguistics and Clinical Psychology (CLPsych 2025)
Federico Ravenda | Fawzia-Zehra Kara-Isitt | Stephen Swift | Antonietta Mira | Andrea Raballo
Proceedings of the 10th Workshop on Computational Linguistics and Clinical Psychology (CLPsych 2025)
Large Language Models are increasingly used in the medical field, particularly in psychiatry where language plays a fundamental role in diagnosis. This study explores the use of open-source LLMs within the MIND framework. Specifically, we implemented a mixed-methods approach for the CLPsych 2025 shared task: (1) we used a combination of retrieval and few-shot learning approaches to highlight evidence of mental states within the text and to generate comprehensive summaries for post-level and timeline-level analysis, allowing for effective tracking of psychological state fluctuations over time (2) we developed different types of ensemble methods for well-being score prediction, combining Machine Learning and Optimization approaches on top of zero-shot LLMs predictions. Notably, for the latter task, our approach demonstrated the best performance within the competition.