Xiaonan Luo

Author directory

2026

Large language models (LLMs) trained with reinforcement objectives often achieve superficially correct answers via shortcut strategies, pairing correct outputs with spurious or unfaithful reasoning and degrading under small input perturbations. We introduce Coherence-Enhanced Policy Optimization (CE-PO), a model agnostic reward fusion framework that augments policy optimization with a differentiable Jacobian coherence signal along the generation pathway from prompt (X) to rationale (Y) to answer (Z). CE-PO estimates internal gradient influence through Jacobian-based sensitivities, uses a reshuffling-based residualization procedure to reduce shortcut-driven reward hacking, and fuses the resulting coherence score with task-accuracy via an Lp combiner that controls the strength of coherence regularization. The unified reward integrates with PPO/GRPO without architectural changes. Across multiple reasoning benchmarks, CE-PO improves accuracy over baselines by 5.49 on average (up to 9.58), while increasing the model’s faithful reasoning capability (12.14 % on average across five backbones).