In this paper, we study a fundamental design problem in pretraining Large Language Models (LLMs) for low-resource language regimes, focusing on cross-family language pairs in which English serves as the high-resource language. Existing works adopt multi-epoch, multi-lingual, and multi-stage training to utilize the limited target-language corpus efficiently, but no prior scaling law can compare recipes spanning these approaches under the same compute budget C and target-language corpus size D_T, leaving the optimal training setup unclear. To address this gap, we propose the M³ Scaling Law, a unified predictive model parameterized by the model scale, the number of target-corpus epochs k, the average target-language ratio r, and the final-stage target-language ratio r_f, which places monolingual single-stage, multi-lingual single-stage, and multi-lingual multi-stage recipes on a single target-language validation-loss surface. Across three language pairs (English paired with Japanese, Indonesian, and Swahili), it extrapolates to unseen hyperparameter regions more accurately than existing scaling laws. Using M³ as a surrogate objective, we derive two practical guidelines for low-resource LLM pretraining: (i) as D_T decreases, the optimal recipe shifts directly from monolingual single-stage to multi-lingual two-stage training at a compute-budget-dependent threshold, with multi-lingual single-stage never optimal in our experimental grid; and (ii) the optimal number of epochs collapses onto a single curve in the scarcity variable D_T/D\*(C), where D\*(C) ∝ C^{𝛼/(𝛼+𝛽)} is the monolingual compute-optimal corpus size.

Encoding graphs as text prompts enables LLM-based graph reasoning. However, effective graph-to-text encodings depend on the downstream task, graph structure, and target LLM, making manual format design time-consuming and brittle. We propose EvoFormat, a black-box evolutionary search framework that automatically discovers interpretable graph-to-text formats represented as human-readable formatting rules over node naming, relation expression, and global arrangement. Using only task-level feedback from LLM outputs, EvoFormat iteratively mutates and selects rules without gradients or internal model access. Across synthetic GraphQA and IMDB-MULTI-derived structural QA tasks over unattributed graphs, EvoFormat outperforms strong hand-crafted encodings in most settings.

Retrieval-Augmented Generation (RAG) enhances large language models (LLMs) by grounding them in external knowledge. However, its application in sensitive domains is limited by privacy risks. Existing private RAG methods typically rely on query-time differential privacy (DP), which requires repeated noise injection and leads to accumulated privacy loss. To address this issue, we propose DP-SynRAG, a framework that uses LLMs to generate differentially private synthetic RAG databases. Unlike prior methods, the synthetic text can be reused once created, thereby avoiding repeated noise injection and additional privacy costs. To preserve essential information for downstream RAG tasks, DP-SynRAG extends private prediction, which instructs LLMs to generate text that mimics subsampled database records in a DP manner. Experiments show that DP-SynRAG achieves superior performance to the state-of-the-art private RAG systems while maintaining a fixed privacy budget, offering a scalable solution for privacy-preserving RAG.