Synthetic post-training pipelines commonly filter generated samples with reward models or holistic LLM judges, often relying on two assumptions: that a sample’s faithfulness can be judged without the exact source that produced it, and that rejected samples can be discarded rather than systematically recovered. We present a controlled study of both assumptions across gate configurations, recovery strategies, and generator scales, using adversarially injected corpora to provide ground-truth failure labels. We find that grounding faithfulness judgments in the exact source evidence improves over post-hoc retrieval for stronger judges and substantially outperforms reward-only filtering; hallucination and reward gates reject largely distinct sample populations supporting the need for both; and adaptive recovery, which diagnoses failures before repairing them, achieves higher yield and recovery rate than naive regeneration. Downstream fine-tuning quality is driven primarily by generator scale, with filtering and recovery providing their clearest gains when smaller generators are used. We release our evaluation corpus and experimental configurations to support reproducible pipeline design.

Domain fine-tuning degrades the safety of large language models: fine-tuned specialists readily comply with harmful prompts framed in domain language. Existing inference-time defenses that mix logits from a safe anchor model require both models to share a vocabulary, which rules them out for the cross-family specialists where safety is most degraded. We present ALIGNBEAM, a training-free method that lifts this restriction by translating anchor probability mass into the target model’s vocabulary token-by-token at each decoding step; a small LLM judge then selects the safest among K candidate continuations. No weights are changed, and the mixing strength can be tuned at deployment without retraining. Across both cross-vocabulary and same-vocabulary evaluation pairs, ALIGNBEAM substantially raises refusal on structured, front-loaded adversarial benchmarks, with a 4.6-6.6 x inference slowdown inside the band of existing inference-time defenses. The results show that early-token refusal behavior can be transferred between model families at inference time, without touching either model’s weights.

Methods for repairing safety alignment in fine-tuned Large Language Models (LLMs) are evolving quickly, but the literature is fragmented across the repair pipeline: each method is typically tested on its own drifted checkpoint, with its own repair implementation, and judged against its own set of baselines. This makes methods hard to compare or select, and calls for a framework that standardizes drift-induction, repair, and evaluation while remaining flexible enough to support new paradigms and architectures. To address this gap, we introduce SAFETUNE, a source-available library that unifies drift-induction, repair, and evaluation behind a standardized pipeline. SAFETUNE hosts four repair paradigms: weight patching, safety-constrained retraining, gradient-based unlearning, and inference-time steering, with implementations audited and debugged against their original papers. A documented intervention guide suggests a starting repair for a checkpoint’s safety-capability profile, with the full pipeline runnable in four lines of code.