As Large Language Models (LLMs) increasingly generate training data for downstream machine learning systems, the quality of this synthetic data becomes a critical security concern. Low-quality synthetic training data can silently poison retrieval systems deployed in security-sensitive contexts such as software issue triage, user support, and threat intelligence matching. We present a multi-dimensional quality assessment protocol for LLM-generated synthetic training data and apply it to a case study involving 13,579 synthetic user reviews generated from GitHub issues across four open-source Android applications. We evaluate 400 stratified samples using an LLM judge (GPT-4o-mini) along a five-point rubric, find that 10.5% of generated reviews fail to meaningfully capture their source issues, and identify systematic failure patterns concentrated in developer-internal issues (continuous integration, refactoring) and sarcastic persona framings. To validate the LLM judge against human annotation, we compute Cohen’s Kappa between one human rater, a second independent human rater, and the LLM judge on 20 stratified reviews. Our results highlight the need for hybrid human-AI protocols when assessing synthetic data quality for security-critical applications.