We argue that robust alignment for non-verifiable tasks is fundamentally a principle generalization problem: reward should be guided by explicit, auditable principles rather than compressed into a learned scalar function. To operationalize this view, we present the Open Rubric System (OpenRS) , a modular, rubrics-based LLM-as-a-Judge framework built around Pairwise Adaptive Meta-Rubrics (PAMR) and lightweight Pointwise Verifiable Rubrics (PVRs). OpenRS is governed by an explicit meta-rubric—not a rubric itself, but a higher-level guideline that specifies how rubrics should be generated, which criteria to prioritize, and which failure modes are non-negotiable—and instantiates concrete adaptive rubrics on the fly by conditioning on the semantic differences between two candidate responses. It then performs criterion-wise pairwise comparisons and aggregates criterion-level preferences externally, avoiding pointwise weighted scalarization while improving discriminability in open-ended settings. Finally, we instantiate OpenRS as reward supervision in pairwise RL training. Empirically, OpenRS achieves state-of-the-art results on four reward-modeling benchmarks and yields stable gains when replacing scalar reward models in end-to-end RL.

Pre-training and fine-tuning framework has become the standard training paradigm for NLP tasks and is also widely used in industrial-level applications. However, there are still a limitation with this paradigm: simply fine-tuning with task-specific objectives tends to converge to local minima, resulting in a sub-optimal performance. In this paper, we first propose a new paradigm: knowledge rekindle, which aims to re-incorporate the fine-tuned expert model into the training cycle and break through the performance upper bounds of experts without introducing additional annotated data. Then we further propose a unified expert-guided pre-training (UEGP) framework for knowledge rekindle. Specifically, we reuse fine-tuned expert models for various downstream tasks as knowledge sources and inject task-specific prior knowledge to pre-trained language models (PLMs) by means of knowledge distillation. In this process, we perform multi-task learning with knowledge distillation and masked language modeling (MLM) objectives. We also further explored whether mixture-of-expert guided pre-training (MoEGP) can further enhance the effect of knowledge rekindle. Experiments and analysis on eight datasets in GLUE benchmark and a industrial-level search re-ranking dataset show the effectiveness of our method.