Yunzhe Hao
Author directory2026
Audio-Reward-R1: Reasoning Reward Model for Unified Audio Understanding and Generation
Hao Gu | Xiaocheng Yang | Jinyang Wu | Hongbo Zhao | Yunzhe Hao | Songcheng Xu | Ye Bai | Zhao You | Shidong Shang
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Hao Gu | Xiaocheng Yang | Jinyang Wu | Hongbo Zhao | Yunzhe Hao | Songcheng Xu | Ye Bai | Zhao You | Shidong Shang
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Multimodal Reward Models are critical for aligning Multimodal Large Language Models with human preferences. However, existing reward models offer limited interpretability and struggle to deliver reliable preference judgments in audio-centric scenarios. To address this gap, we propose Audio-Reward-R1, a reasoning-based reward model that supports both audio understanding and audio generation preference tasks. Audio-Reward-R1 formulates reward modeling as a structured reasoning process, producing dimension-level analyses prior to the final preference judgment. To enable reliable reasoning supervision, we construct a high-quality reasoning-enhanced audio preference dataset with structured reasoning traces aligned with preference annotations. Audio-Reward-R1 is first trained via supervised fine-tuning as a cold-start on this reasoning-enhanced dataset, followed by reinforcement learning with verifiable rewards. Finally, we introduce Audio-Reward-Bench, a human-verified evaluation benchmark for evaluating audio reward models across both audio understanding and audio generation preference tasks. Extensive experimental results demonstrate the effectiveness and reliability of Audio-Reward-R1.