Qinlin Zhao

Author directory

2026

As LLMs evolve from standalone assistants into autonomous agents, they increasingly operate in populations where communication, coordination, and competition form community-like interaction networks. In such settings, individually value-steered behaviors can accumulate into group-level dynamics, raising the question of how human values shape collective outcomes in interacting LLM-agent populations. We introduce CIVA, a community-level value analysis framework that varies value direction and prevalence, then measures how these interventions propagate through repeated interactions into macro-level dynamics and micro-level behaviors. We apply it in a resource-constrained multi-agent community where LLM agents autonomously communicate, cooperate, and compete. The value scan reveals distinct structural response patterns: activation-supporting values such as benevolence supports persistence and resilience, an activation-constraining value such as tradition regulates growth and cohesion, and a bidirectionally disruptive value such as power perturbs collective regimes in both directions. At the micro level, representative interventions induce emergent behaviors including deception, betrayal, self-preservation, and power-seeking. These results show that human values can act as population-level structural variables in agent communities and motivate evaluation for multi-agent value alignment.
We introduce ScreenHaystack, a dynamic needle-in-a-haystack benchmark for evaluating spatial reliability in GUI grounding. Instead of testing each target at a fixed position, ScreenHaystack systematically relocates controlled target icons across high-resolution GUI backgrounds and measures whether models can localize them consistently. Using this benchmark, we find that leading GUI grounding models, including Qwen3-VL, UI-TARS, GTA, and UI-Venus, exhibit blind zones: spatial regions where grounding accuracy drops sharply despite fixed target appearance and instruction. These blind zones transfer to unseen ScreenSpot-Pro examples: targets inside blind zones are consistently harder to ground, with Qwen3-VL dropping by 16.1 percentage points, and controlled relocation shows that moving targets into blind zones decreases accuracy while moving them out improves accuracy. We further show through controlled synthetic experiments that uneven spatial coverage in training data can induce such blind zones. Therefore, we propose a simple strategy, blind-zone-oriented augmentation, which adds supervision in blind zones and improves ScreenSpot-Pro accuracy over both original and random augmented Click-100k fine-tuning.
Measuring the value orientations of LLMs is helpful for diagnosing model risk and understanding their misalignment behavior. However, existing value benchmarks are often AI-generated and lack validity verification, leading to noisy and unreliable results. Inspired by Psychometrics, we propose a novel Subset Item Response Theory (S-IRT) model for assessing LLMs’ values under any value theories. S-IRT directly models the subset of values jointly expressed in each LLM response, rather than reducing responses to binary outcomes as in traditional IRT-based ability assessment. Meanwhile, it jointly estimates the LLM’s latent values and captures the relative priorities among them. Based on it, we further design a tailored information reward that estimates the information gain of each item in reducing uncertainty about the evaluated model’s value, and thus helps select a small set of high-quality items from existing benchmarks. Extensive experiments on 23 popular LLMs and three value systems (21 value dimensions) demonstrate S-IRT can be applied to multiple value systems simultaneously and improve evaluation reliability and validity. The information reward also helps reduce the number of questions to as few as 200 while preserving comparable evaluation effectiveness.

2025

Large Language Models (LLMs) are increasingly deployed as human assistants across various domains where they help to make choices. However, the mechanisms behind LLMs’ choice behavior remain unclear, posing risks in safety-critical situations. Inspired by the intrinsic and extrinsic motivation framework within the classic human behavioral model of Self-Determination Theory and its established research methodologies, we investigate the factors influencing LLMs’ choice behavior by constructing a virtual QA platform that includes three different experimental conditions, with four models from GPT and Llama series participating in repeated experiments. Our findings indicate that LLMs’ behavior is influenced not only by intrinsic attention bias but also by extrinsic social influence, exhibiting patterns similar to the Matthew effect and Conformity. We distinguish independent pathways of these two factors in LLMs’ behavior by self-report. This work provides new insights into understanding LLMs’ behavioral patterns, exploring their human-like characteristics.

2024

Peer review is fundamental to the integrity and advancement of scientific publication. Traditional methods of peer review analyses often rely on exploration and statistics of existing peer review data, which do not adequately address the multivariate nature of the process, account for the latent variables, and are further constrained by privacy concerns due to the sensitive nature of the data. We introduce AgentReview, the first large language model (LLM) based peer review simulation framework, which effectively disentangles the impacts of multiple latent factors and addresses the privacy issue. Our study reveals significant insights, including a notable 37.1% variation in paper decisions due to reviewers’ biases, supported by sociological theories such as the social influence theory, altruism fatigue, and authority bias. We believe that this study could offer valuable insights to improve the design of peer review mechanisms.