Agentic recursive training systems increasingly reuse model-generated tasks, proxy feedback, and selected trajectories to refine policies over multiple rounds. This creates feedback loops in which small conditional biases can accumulate across training and alter later selection and updates. We study whether trigger-conditional proxy-feedback biases can produce emergent backdoors in a controlled recursive training loop. A Curriculum Agent proposes tasks, an Executor Agent generates rollouts, and environment feedback is selectively biased on rare markers, affecting reward, selection, and curriculum updates. Under matched biased supervision, Reward Hacking + SFT and Reward Hacking + RLHF reach 31.27% and 53.85% ASR, while the full loop reaches 97.62% at comparable clean accuracy. The effect persists across Qwen3-4B, LLaMA-3-8B, BLIP-2-OPT-2.7B, and Qwen2.5-14B, transfers to GSM8K, BBH, and HotpotQA, extends to a tool-augmented setting, and is only partially reduced by post-hoc defenses. These results suggest that feedback and curriculum mechanisms in agentic recursive training should be audited for long-horizon bias amplification.

Transformer models face scalability challenges in causal language modeling (CLM) due to inefficient memory allocation for growing key-value (KV) caches, which strains compute and storage resources. Existing methods like Grouped Query Attention (GQA) and token-level KV optimization improve efficiency but rely on rigid resource allocation, often discarding “low-priority” tokens or statically grouping them, failing to address the dynamic spectrum of token importance. We propose mixSGA, a novel mixture-of-expert (MoE) approach that dynamically optimizes token-wise computation and memory allocation. Unlike prior approaches, mixSGA retains all tokens while adaptively routing them to specialized experts with varying KV group sizes, balancing granularity and efficiency. Our key novelties include: (1) a token-wise expert-choice routing mechanism guided by learned importance scores, enabling proportional resource allocation without token discard; (2) weight-sharing across grouped attention projections to minimize parameter overhead; and (3) an auxiliary loss to ensure one-hot routing decisions for training-inference consistency in CLMs. Extensive evaluations across Llama3, TinyLlama, OPT, and Gemma2 model families show mixSGA’s superiority over static baselines. On instruction-following and continued pretraining tasks, mixSGA achieves higher ROUGE-L and lower perplexity under the same KV budgets.