While Large Language Models (LLMs) demonstrate remarkable capabilities in open-ended creative generation, they notably struggle with Format-Constrained Generation tasks—such as poetry and lyrics—where strict adherence to multidimensional structural constraints (i.e., format, phonetics, and rhyme) is prerequisite to aesthetic value. Existing paradigms predominantly rely on unreliable prompting or rigid constrained decoding strategies; the former often fails to ensure compliance, while the latter compromises inference latency and disrupts the natural probability distribution, degrading generation quality. To bridge this gap, we establish CCP-Arena, a rigorous testbed for Chinese Classical Poetry, and proposeProgressive Structural Internalization (PSI) a novel framework designed to embed external constraints into the model’s intrinsic intuition. PSI initiates withStructural Scaffolding via Explicit Cognitive Planning, utilizing explicit template to provide a structural scaffold for subsequent generation. This is followed by a Cascaded Reinforcement Learning stage guided by a Holistic Reward Model, which optimizes for precise structural-semantic alignment. Extensive experiments demonstrate that PSI achieves state-of-the-art performance, surpassing baselines in both strict constraint adherence and literary aesthetics. Furthermore, mechanistic analysis confirms that our method effectively internalizes structural information into the model’s latent representations, offering a robust and efficient solution for constrained creative generation.

Language can produce many expressions from a limited set of units because linguistic units participate in two basic relations. Syntagmatic relations determine which units can combine, while paradigmatic relations determine which units can substitute for one another in the same position. We study whether explicitly learning these relations helps language models generalize from limited data. We introduce two auxiliary objectives for masked language modeling. The Syntagmatic objective aligns a masked representation with relevant context selected by attention, while the Paradigmatic objective aligns it with the embedding neighbors of the target token. Both objectives are generated by the model itself and require no additional data, annotation, or trainable parameters. Experiments on BabyLM 2026 Strict-Small show that both single-relation systems improve mean Overall performance over Standard MLM, with Paradigmatic giving the most consistent gains across three seeds. Relay Joint combines the two objectives and shows a strong neighborhood-alignment signal at its submitted checkpoint, although its downstream gains vary across seeds. These results suggest that relational objectives can provide useful learning signals under limited data, while stronger internal alignment does not always lead to more stable downstream performance. Code is now publicly available on GitHub.