Large language models (LLMs) are highly sensitive to prompts, but most automatic prompt optimization (APO) methods assume access to ground-truth references (e.g., labeled validation data) that are costly to obtain. We propose the Prompt Duel Optimizer (PDO), a sample-efficient framework for label-free prompt optimization based on pairwise preference feedback from an LLM judge. PDO casts prompt selection as a dueling-bandit problem and combines (i) Double Thompson Sampling to prioritize informative comparisons under a fixed judge budget, with (ii) top-performer guided mutation to expand the candidate pool while pruning weak prompts. Experiments on BIG-bench Hard (BBH) and MS MARCO show that PDO consistently identifies stronger prompts than label-free baselines, while offering favorable quality–cost trade-offs under constrained comparison budgets.

Large-scale content moderation requires models that can apply complex, evolving policies under limited expert supervision. We present an empirical study of reinforcement learning (RL) for policy-grounded moderation across three production-derived tasks. Using established RL and inference primitives, we develop three integrity-specific designs: a policy-grounded full-trace reward, rationale-marginalized Monte Carlo scoring, and policy-structured reflection. Each design addresses an observed failure mode, including reasoning-length collapse and polarized confidence estimates. Across the three tasks, RL-Only systems trained on 200–836 labeled examples remain competitive with supervised fine-tuning (SFT) systems trained on thousands of examples, corresponding to up to approximately one order of magnitude greater label efficiency at the evaluated operating points. Performance generally improves with additional data and training compute before saturating, although larger rollout counts exhibit variability. The strongest SFT→RL systems attain point-estimate performance comparable to policy-trained full-time employee (FTE) reviewers across the three tasks.

Due to the limited context window, Large Language Models (LLMs) struggle with processing long contexts. Although fine-tuning can extend the context window, it incurs substantial computation costs. In contrast, recent tuning-free approaches reallocate the attention mechanism or incorporate temporary trainable parameters. In this work, by jointly modeling instance-level generation with a limited context window and learning over sequential data, we rethink the long context generation of LLMs from a continual learning perspective. In practice, we inspect existing representative approaches and analyze their synergy with continual learning strategies. Moreover, we integrate these strategies into current approaches to further boost LLMs’ efficiency in processing long contexts. Comprehensive experiments and analysis confirm the feasibility of continual learning insights for improving long-context processing.