Multi-chapter novel generation requires models to balance two quality dimensions simultaneously: chapter-level reader engagement and global narrative rhythm. Existing approaches have mainly focused on planning, memory, and execution, while paying limited attention to the mismatch between training signals and reader experience. We propose **ChapterRL**, which reformulates multi-chapter generation as a reward design problem. Specifically, ChapterRL introduces two complementary reward signals. **Reading Depth Residual** (RDR) derives debiased chapter-level quality signals from reader continuation and completion behavior through genre–position stratified normalization. **Dynamic Rhythm Trajectory** (DRT) uses an arousal estimator to impose a threshold-gated penalty on systematic rhythm collapse in the later part of a story. The two signals correspond to chapter attractiveness and global narrative momentum, and are jointly integrated into reinforcement learning training. We evaluate the full ChapterRL policy on three automatic benchmarks and separately validate its RDR reward model on task-aligned narrative domains of the Chinese WritingPreferenceBench. Results show that ChapterRL improves chapter quality and rhythm stability over supervised fine-tuning and single-reward variants, while RDR exhibits strong narrative preference alignment as an efficient scalar reward model.Code is available at [ChapterRL](https://github.com/SorBetQ/ChapterRL).