Several high-stakes administrative tasks require supporting verdicts and decisions with auditable reasoning. While the application of LLMs in this regard has been studied, existing benchmarks are unsuitable because they either score only the final answer or evaluate citation quality separately from outcome correctness. We introduce HS-BENCH, a benchmark for customs classification, to study this problem. The task involves assigning tariff codes to products with reasoning grounded in legal authorities. HS-BENCH consists of 1,390 product descriptions paired with supporting legal authorities across diverse sources, including dictionary definitions, statutory notes, and prior rulings. We propose Strict Support Accuracy (SSA), a metric that credits a prediction only when both the outcome and the reasoning are well supported. Evaluating frontier models in closed-book and oracle-evidence regimes, we find that classification accuracy and SSA diverge sharply: models often reach the correct outcome through incorrect reasoning. We release the dataset, extraction pipeline, and evaluation code at https://github.com/adalat-ai-tech/hs-bench to support research on auditable LLM systems in administrative law.