Classical Chinese poetry generation and evaluation with Large Language Models (LLMs) face two intertwined challenges: existing generation benchmarks rely on simplistic keyword inputs devoid of literary context, and no ground-truth benchmark exists for objectively validating poetry evaluation methods. We present a comprehensive framework addressing both challenges. On the **generation** side, we construct **Honglou-Poem**, the first narrative-driven poetry generation benchmark derived from Dream of the Red Chamber, where each entry provides rich author personas and narrative backgrounds—moving far beyond traditional keyword-to-poem paradigms. A ”Digital Poetry Party” pipeline evaluates 12 state-of-the-art LLMs in this challenging setting. On the **evaluation** side, we introduce **Poetry-Tree**, a benchmark with axiomatically ordered quality levels created through systematic merit degradation, providing objective ground truth for evaluator validation. Leveraging Poetry-Tree as a training signal, we employ **GRPO** reinforcement learning with Spearman’s correlation as the reward to train a dedicated poetry evaluation model, **Poet-4B**. Our Poet-4B achieves a Spearman’s 𝜌 of **0.8253**, dramatically improving from the base model’s 0.2929 and closely approaching the best closed-source model GLM-4.6 (0.8764) with over 100× fewer parameters—a gap of only 0.051 in Spearman’s 𝜌. We release all benchmarks, trained models, and a leaderboard to facilitate future research.

The performance of Large Language Models (LLMs) is significantly sensitive to the contextual position of information in the input. To investigate the mechanism behind this positional bias, our extensive experiments reveal a consistent phenomenon we term the attention basin: when presented with a sequence of structured items (e.g., retrieved documents or few-shot examples), models systematically assign higher attention to the items at the beginning and end of the sequence, while neglecting those in the middle. Crucially, our analysis further reveals that allocating higher attention to critical information is key to enhancing model performance. Based on these insights, we introduce Attention-Driven Reranking (AttnRank), a two-stage framework that (i) estimates a model’s intrinsic positional attention preferences using a small calibration set, and (ii) reorders retrieved documents or few-shot examples to align the most salient content with these high-attention positions. AttnRank is a model-agnostic, training-free, and plug-and-play method with minimal computational overhead. Experiments on multi-hop QA and few-shot in-context learning tasks demonstrate that AttnRank achieves substantial improvements across 10 large language models of varying architectures and scales, without modifying model parameters or training procedures.