Xuyang Cao

Author directory

Other people with similar names: Xuyang Cao


2026

In multilingual pretraining, the test loss of a pretrained model is heavily influenced by the proportion of each language in the pretraining data, namely the *language mixture ratios*. Multilingual scaling laws can predict the test loss under different language mixture ratios and can therefore be used to estimate the optimal ratios. However, the current approaches to multilingual scaling laws do not explicitly quantify *cross-lingual transfer* between individual languages, resulting in suboptimal mixture-ratio optimization. In this paper, we consider multilingual pretraining as a cooperative game in which each language acts as a player that jointly contributes to pretraining, gaining the resulting reduction in test loss as the payoff. Consequently, from the perspective of cooperative game theory, we quantify the cross-lingual transfer from each language by its contribution in the game, and propose a game-theoretic multilingual scaling law called *ShapleyLaw*. Our experiments show that ShapleyLaw outperforms baseline methods in model performance prediction and language mixture optimization. Our code and data are available at [https://github.com/YcTtXyLLM/ShapleyLaw](https://github.com/YcTtXyLLM/ShapleyLaw).