Dimitris Bilianos

Author directory

2026

This paper describes the participation of the dbilianos team in the ArGuard 2026 Shared Task on harmful prompt detection for Arabic LLM safety evaluation. We participated in both subtasks: binary safe/unsafe classification (Task B1) and granular harm category classification (Task B2). Our approach systematically tested a central hypothesis: whether translating Arabic prompts into English improves classification performance. We experimented with multiple pipelines, including classical machine learning with TF-IDF features, zero-shot and few-shot prompting with DeepSeek-V4, and hybrid approaches. Our results reveal a divergence across subtasks. For binary classification, DeepSeek zero-shot on English translations achieved 0.89 macro-F1, securing 4th place in this subtask, and confirming that LLM-based prompting is highly effective for coarse-grained safety decisions. For granular classification, however, the best performance came from training a Support Vector Machine directly on original Arabic text using TF-IDF features, which achieved 0.61 macro-F1, placing the team 16th in this subtask. This outperformed our translation-based strategies, suggesting that fine-grained harm categorization is highly sensitive to linguistic and cultural nuance that is lost or distorted during translation. The translation hypothesis was only partially supported locally, and ultimately reversed on the final test set for both subtasks. We discuss implications for cross-lingual safety evaluation and highlight the limitations of translation-based pipelines for tasks requiring subtle intent interpretation.