Quality-aware synthetic-data selection assumes that examples rated highly by an LLM judge will help downstream learning. We test this assumption in a controlled replay across two low-resource African-language classification tasks, four languages, and seven selectors. Six selectors retain 64 examples per task-language cell. CoSDA-V2 yields the highest judged label correctness (0.904 versus 0.767 for naive) and the lowest measured shortcut score, but AlpaGasus leads CoSDA-V2 on downstream Macro-F1 in both the main replay (0.202 versus 0.163) and the three-seed refit (0.193 versus 0.157).

Adding the audit’s strict gate to AlpaGasus reduces Macro-F1 by 0.029, driven by one cell where only 31 of 64 candidates survive. Because the gate changes pool composition and training-set size, this comparison measures their joint effect. LESS also falls below naive when unconstrained top-k collapses label balance, but recovers with quotas. At this budget, retention constraints account for more of the observed differences than a consistent ranking advantage. In this replay, audit scores do not reliably predict downstream utility. We therefore recommend reporting both metrics for the same retained sets.