Training Large Language Model (LLM) agents via Reinforcement Learning (RL) in federated settings faces fundamental challenges, including model heterogeneity across clients, prohibitive communication costs and severe data heterogeneity. We propose a semantic federated learning framework called SemFed that addresses all challenges by communicating behavioral patterns rather than model parameters. Each client extracts compact semantic policy summaries from local training trajectories, which are aggregated at the server into shared, elite and anti-patterns. These aggregated patterns are injected back into local training through different complementary channels. Since the communicated objects are structured behavioral abstractions, SemFed supports heterogeneous model architectures while reducing per-round communication to kilobytes. Experiments on ALFWorld and WebShop across multiple heterogeneous LLM backbones demonstrate that SemFed consistently surpasses strong federated baselines in task success while reducing communication overhead by orders of magnitude.

Hypernetworks are a class of meta-networks that generate weights for main neural networks. Their unique parameter spaces necessitate exploring suitable optimization strategies to enhance performance, especially for language models. However, a comprehensive investigation into optimization strategies for hypernetworks remains absent. To address this gap, we analyze the loss landscape of hypernetworks and propose that restart optimization strategies can improve their performance for language models. We find that hypernetworks have inherently more complicated loss landscapes compared to conventional networks due to their distinct parameter spaces. Consequently, a restart strategy that periodically resets the learning rate can facilitate better convergence for hypernetworks. Through experiments on instruction tuning and multi-task training, we demonstrate that the restart strategy consistently enhances the performance of hypernetworks for language models, often more effectively than for conventional deep neural networks. Our findings highlight the importance of tailored optimization techniques to unlock the full potential of hypernetworks in natural language processing tasks.