Daniel M. Bikel

Author directory

Other people with similar names: Daniel M. Bikel


2026

LLM benchmark labels are frozen at release and silently copied into downstream benchmarks, errors and all. Our Item Response Theory indicator surfaces these mislabels at 95% precision in the top 200, across 114 models on seven preference and multiple-choice benchmarks, outperforming a supervised classifier. We trace these errors to mechanical labeling rules, upstream errors inherited unchanged from source datasets, and ambiguous items with no defensible single key. The same fit reveals that reward models specialize in stylistic preference over factual knowledge, and exposes one frontier reward model that agrees with detected mislabels at 78% accuracy versus 38% for its peers, consistent with contamination or benchmark-specific over-optimization.