Shuyuan Zheng
Author directory2026
Mechanistic Diagnostics of Spatial Lexical Bias in Multimodal Large Language Model Spatial Reasoning
Chuang Ma | Qianying Liu | Tomoyuki Obuchi | Fei Cheng | Wang Yang | Sudong Cai | Shuyuan Zheng | Akiko Aizawa | Sadao Kurohashi
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Chuang Ma | Qianying Liu | Tomoyuki Obuchi | Fei Cheng | Wang Yang | Sudong Cai | Shuyuan Zheng | Akiko Aizawa | Sadao Kurohashi
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Multimodal large language models (MLLMs) remain unreliable on spatial multiple-choice questions, and their failures are often attributed to poorly attended visual information. We identify a complementary failure mode, spatial lexical bias: a spatial relation word added to the answer options can act as a lexical-semantic distractor that draws the model’s decision toward that option. Using nine open-weight MLLMs, we show that this phenomenon is widespread. We then isolate diagnostic cases in which a model answers a binary spatial question correctly yet consistently chooses a newly added third spatial option, which we call binary-stable but ternary-fragile cases. Leveraging mechanistic interpretability tools on these cases, we find that the failure arises on the language side rather than the visual side: visual attention analyses and residual-stream probes show the correct spatial relation remains internally available, while irrelevant-option controls, activation patching, and sparse component interventions trace the bias to specific LLM-side channels and neurons. Accordingly, we show that a lightweight LLM-only DPO update on tiny single-object-pair synthetic data mitigates the bias, lifting four-way robust accuracy by up to 100 points on synthetic data, and by 68.0, 32.6, and 20.1 points on broader evaluation datasets WhatsUp, SpatialMQA-Direct, and VSR.
ShapleyLaw: A Game-Theoretic Approach to Multilingual Scaling Laws
Xuyang Cao | Qianying Liu | Chuan Xiao | Yusuke Oda | Jiayi Wang | Pontus Stenetorp | Daisuke Kawahara | Makoto Onizuka | Sadao Kurohashi | Shuyuan Zheng
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Xuyang Cao | Qianying Liu | Chuan Xiao | Yusuke Oda | Jiayi Wang | Pontus Stenetorp | Daisuke Kawahara | Makoto Onizuka | Sadao Kurohashi | Shuyuan Zheng
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
In multilingual pretraining, the test loss of a pretrained model is heavily influenced by the proportion of each language in the pretraining data, namely the *language mixture ratios*. Multilingual scaling laws can predict the test loss under different language mixture ratios and can therefore be used to estimate the optimal ratios. However, the current approaches to multilingual scaling laws do not explicitly quantify *cross-lingual transfer* between individual languages, resulting in suboptimal mixture-ratio optimization. In this paper, we consider multilingual pretraining as a cooperative game in which each language acts as a player that jointly contributes to pretraining, gaining the resulting reduction in test loss as the payoff. Consequently, from the perspective of cooperative game theory, we quantify the cross-lingual transfer from each language by its contribution in the game, and propose a game-theoretic multilingual scaling law called *ShapleyLaw*. Our experiments show that ShapleyLaw outperforms baseline methods in model performance prediction and language mixture optimization. Our code and data are available at [https://github.com/YcTtXyLLM/ShapleyLaw](https://github.com/YcTtXyLLM/ShapleyLaw).
2025
Legal Fact Prediction: The Missing Piece in Legal Judgment Prediction
Junkai Liu | Yujie Tong | Hui Huang | Bowen Zheng | Yiran Hu | Peicheng Wu | Chuan Xiao | Makoto Onizuka | Muyun Yang | Shuyuan Zheng
Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing
Junkai Liu | Yujie Tong | Hui Huang | Bowen Zheng | Yiran Hu | Peicheng Wu | Chuan Xiao | Makoto Onizuka | Muyun Yang | Shuyuan Zheng
Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing
Legal judgment prediction (LJP), which enables litigants and their lawyers to forecast judgment outcomes and refine litigation strategies, has emerged as a crucial legal NLP task. Existing studies typically utilize legal facts, i.e., facts that have been established by evidence and determined by the judge, to predict the judgment. However, legal facts are often difficult to obtain in the early stages of litigation, significantly limiting the practical applicability of fact-based LJP. To address this limitation, we propose a novel legal NLP task: legal fact prediction (LFP), which takes the evidence submitted by litigants for trial as input to predict legal facts, thereby empowering fact-based LJP technologies to make predictions in the absence of ground-truth legal facts. We also propose the first benchmark dataset, LFPBench, for evaluating the LFP task. Our extensive experiments on LFPBench demonstrate the effectiveness of LFP-empowered LJP and highlight promising research directions for LFP.
2024
Shall We Team Up: Exploring Spontaneous Cooperation of Competing LLM Agents
Zengqing Wu | Run Peng | Shuyuan Zheng | Qianying Liu | Xu Han | Brian I. Kwon | Makoto Onizuka | Shaojie Tang | Chuan Xiao
Findings of the Association for Computational Linguistics: EMNLP 2024
Zengqing Wu | Run Peng | Shuyuan Zheng | Qianying Liu | Xu Han | Brian I. Kwon | Makoto Onizuka | Shaojie Tang | Chuan Xiao
Findings of the Association for Computational Linguistics: EMNLP 2024
Large Language Models (LLMs) have increasingly been utilized in social simulations, where they are often guided by carefully crafted instructions to stably exhibit human-like behaviors during simulations. Nevertheless, we doubt the necessity of shaping agents’ behaviors for accurate social simulations. Instead, this paper emphasizes the importance of spontaneous phenomena, wherein agents deeply engage in contexts and make adaptive decisions without explicit directions. We explored spontaneous cooperation across three competitive scenarios and successfully simulated the gradual emergence of cooperation, findings that align closely with human behavioral data. This approach not only aids the computational social science community in bridging the gap between simulations and real-world dynamics but also offers the AI community a novel method to assess LLMs’ capability of deliberate reasoning.Our source code is available at https://github.com/wuzengqing001225/SABM_ShallWeTeamUp