Yitong Ma
Author directory2026
DualEval: Joint Model-Item Calibration for Unified LLM Evaluation
Aaron J. Li | Hao Huang | Youngmin Park | Yitong Ma | Wei-Lin Chiang | Li Chen | Cho-Jui Hsieh | Bin Yu | Ion Stoica
Findings of the Association for Computational Linguistics: EMNLP 2026
Aaron J. Li | Hao Huang | Youngmin Park | Yitong Ma | Wei-Lin Chiang | Li Chen | Cho-Jui Hsieh | Bin Yu | Ion Stoica
Findings of the Association for Computational Linguistics: EMNLP 2026
Current LLM evaluation relies on two complementary but often disconnected signals: static benchmarks with objective correctness labels and arena-style preference data that better reflect open-ended user interactions. We introduce DualEval, a latent model-item calibration framework that represents models and evaluation items in a shared space, jointly estimating model ability together with item difficulty and sharpness. We apply DualEval across four domains: coding, math, miscellaneous domain-knowledge tasks, and generic everyday user queries. Our evaluation uses 18 frontier LLMs, static benchmark labels, and reward-model scores validated against held-out human preferences for open-ended model responses. Empirically, our framework produces reliable and balanced model rankings, and its learned item-level profiles support downstream applications such as benchmark compression for sample-efficient evaluation and anomaly detection for contamination or outlier analysis. Overall, DualEval unifies static and arena-style evaluation through joint model-item calibration, producing model rankings and item-level diagnostics that support more sample-efficient, interpretable, and auditable evaluation pipelines.