While Large Language Models (LLMs) have been tested for hate speech detection, their reliability in subjective annotation settings remains underexplored. Existing improved prompting approaches for hate speech classification often rely on continuous context retrieval, which can be computationally expensive and unnecessary for confident predictions. To address this, we propose an uncertainty-aware prompting pipeline that integrates Uncertainty Quantification (UQ) into LLM-based hate speech classification. The framework uses uncertainty scores to identify ambiguous and potentially unreliable predictions through threshold-based filtering, enabling selective application of advanced context-enriched prompting only when needed. We evaluate the approach on two benchmark datasets—Measuring Hate Speech (Sachdeva et al., 2022) and CREHate (Lee et al., 2024—across five LLMs using two advanced prompting strategies: Persona-Aware and Annotation-Grounded Few-Shot Prompting. Experimental results show that the proposed pipeline consistently improves classification performance and reduces uncertainty, leading to more reliable model behavior with improved computational efficiency.