Reinforcement learning has driven the largest post-training gains on tasks with verifiable rewards, such as code and math. Extending this success to subjective tasks remains difficult: good responses must satisfy multiple soft criteria simultaneously, and no scalar reward can indicate which aspect of a response should improve. Personalization is a clear instance of this. A personalized response must balance helpfulness, user-specific relevance, and naturalness, yet standard verifiers cannot localize which specific dimension needs improvement. In this regime, supervised fine-tuning quickly saturates, and PPO with Bradley-Terry reward models is prone to reward hacking, especially via length bias, producing verbose or superficially personalized responses. We propose Critique-Post-Edit, an RL framework that combines (1) a personalized Generative Reward Model (GRM) producing multi-dimensional scores and natural-language critiques, and (2) a critique-guided editing step that turns feedback into nearby alternative responses for policy learning. The reward signal stays scalar; our contribution is to use personalized critiques to construct more informative local alternatives (responses close to the on-policy rollout in edit distance) during training. Under length-controlled evaluation, Critique-Post-Edit improves over PPO by 11.0 and 11.6 win-rate points on Qwen2.5-7B and 14B models, respectively. Notably, the 14B model matches or outperforms GPT-4.1 on the evaluated benchmarks. A static mechanism study further shows that critique-guided edits stay more local than extra rollouts and shift scores more favorably than generic rewrites. All of our experiments are on personalization; systematic evaluation on other subjective tasks is left to future work.

The rapid advancement of Large Language Models (LLMs) has led to the development of benchmarks that consider temporal dynamics, however, there remains a gap in understanding how well these models can generalize across temporal contexts due to the inherent dynamic nature of language and information. This paper introduces the concept of temporal generalization in LLMs, including bias in past and future generalizations. Then we introduce FreshBench, a new evaluation framework that employs fresh text and event prediction for assessing LLMs’ temporal adaptability, ensuring the evaluation process free from data leakage and subjective bias. The experiment shows significant temporal biases and a decline in performance over time.

Multimodal large language models (MLLMs) have broadened the scope of AI applications. Existing automatic evaluation methodologies for MLLMs are mainly limited in evaluating objective queries without considering real-world user experiences, inadequately addressing the nuances of creative and associative multimodal tasks. However, the open-ended and subjective nature of such tasks poses a significant challenge to the evaluation methodology, where it is difficult to define the ground-truth answers for them. To this end, in our paper, we propose a new evaluation paradigm for MLLMs, which is evaluating MLLMs with per-sample criteria using potent MLLM as the judge. To validate the feasibility and effectiveness of this paradigm, we design a benchmark, dubbed MLLM-Bench, by curating the evaluation samples across six comprehensive cognitive levels. We benchmark 26 popular MLLMs in a pairwise-comparison fashion, showing diverse performance across models. Moreover, the validity of our benchmark manifests itself in reaching 88.02% agreement with human evaluation. We contend that the proposed paradigm explores the potential of MLLMs as effective evaluation tools with the help of per-sample criteria.