Effective prompt engineering remains a central challenge in fully harnessing the capabilities of LLMs. While well-designed prompts can dramatically enhance performance, crafting them typically demands expert intuition and a nuanced understanding of the task. Moreover, the most impactful prompts often hinge on subtle semantic cues, ones that may elude human perception but are crucial for guiding LLM behavior. In this paper, we introduce PRL (Prompts from Reinforcement Learning), a novel RL-based approach for automatic prompt generation. Unlike most prior methods, PRL can also generate novel few-shot demonstrations rather than selecting only from examples seen during training. Empirically, our approach achieves state-of-the-art performance across a diverse set of benchmarks, including text classification, text simplification, and mathematical reasoning, while remaining competitive on text summarization. Our code is available at https://github.com/Batorskq/prl.