Automatic harmfulness metrics and LLM-based judges are central to LLM safety assessment, yet differences in evaluation criteria, methodologies, and output formats may yield inconsistent judgments for the same model response. The lack of systematic research on these evaluators undermines the credibility and consistency of LLM safety assessments. To bridge this gap, we introduce HarmMetric Eval, a benchmark for both overall and fine-grained assessment of harmfulness evaluators. We construct a dataset of representative harmful prompts paired with diverse harmful responses and non-harmful counterparts across multiple categories. We propose a flexible scoring mechanism that rewards metrics for correctly ranking harmful responses above non-harmful ones, making it applicable to nearly all metrics and judges with varying output formats and scoring scales. Using HarmMetric Eval, we find that conventional reference-based metrics such as ROUGE and METEOR can outperform existing LLM-based judges in fine-grained harmfulness evaluation, challenging prevailing assumptions about LLM superiority. Fine-grained analysis reveals that LLM judges tend to overestimate irrelevant and useless responses. Furthermore, we build a new harmfulness judge by incorporating fine-grained criteria into its prompt template and leveraging reference-based metrics to fine-tune its base LLM. The resulting judge demonstrates leading performance on HarmMetric Eval and two external human-labeled datasets.

Knowledge injection updates pretrained MLLMs with new factual or domain-specific knowledge, but fitting full authoritative answers can cause drift in non-updated behavior. Online distillation limits this drift by training on model-generated rollouts, yet uniform reference-conditioned distillation gives coarse supervision: it can under-emphasize reference-supported rollout tokens and supervise omitted facts only indirectly. We introduce RoCo-ACE, a rollout-conditioned online distillation objective for knowledge injection: RoCo uses same-rollout reference-free/reference-conditioned likelihood contrast to reallocate additional distillation weight to reference-supported rollout tokens, while ACE adds sparse reference-side anchored correction for authoritative anchors omitted from the rollout, without full-answer imitation. Across three knowledge-injection settings, six retention benchmarks, multiple baselines and base models, RoCo-ACE achieves the best injected-knowledge accuracy among compared methods while keeping evaluated retention close to the base model.