Guangxu Chen

Author directory

2026

Hidden states of large language models (LLMs) are typically analyzed as flat Euclidean vectors through probing, anisotropy, and intrinsic-dimension lenses. Yet much of what these models process (taxonomies, program structure, multi-step derivations) is hierarchical, and whether hidden-state distances reflect such hierarchy remains untested. We present the first systematic study of Gromov hyperbolicity (GH), a distance-based measure of tree-likeness, in prompt-token hidden states of open-weight LLMs. Across model families, scales, layer depths, and domains spanning math, code, commonsense, and truthfulness benchmarks, we construct a hyperbolicity atlas charting the normalized statistic 𝛿rel along each axis. Four findings emerge: (i) depth dominates, with middle layers forming a high-𝛿rel plateau and final layers dropping toward tree-like structure; (ii) scale is weak and non-monotonic; (iii) matched-scale families occupy distinct regimes, with specialization gaps exceeding scale effects; and (iv) domains interact with specialization, e.g., HumanEval shifts from the highest- to the lowest-𝛿rel domain between base and coder checkpoints. GH thus serves as a lightweight diagnostic that localizes where hierarchical distance structure emerges and which comparisons warrant deeper analysis.
As code large language models (LLMs) evolve into tool-interactive agents via the Model Context Protocol (MCP), their generalization is increasingly limited by low-quality synthetic data and the diminishing returns of quantity scaling; moreover, quantity-centric scaling exhibits an early bottleneck that underutilizes trajectory data. We propose TDScaling, a Trajectory Diversity Scaling-based data synthesis framework for code agents that scales performance through diversity rather than raw volume. Moreover, TDScaling is more data-efficient: under a fixed training budget, increasing trajectory diversity yields larger gains than adding more trajectories, improving the performance-cost trade-off for agent training. TDScaling integrates four innovations: (1) a Business Cluster mechanism that captures real-service logical dependencies; (2) a Blueprint-driven multi-agent paradigm that enforces trajectory coherence; (3) an adaptive evolution mechanism that steers synthesis toward long-tail scenarios using Domain Entropy, Reasoning Mode Entropy, and Cumulative Action Complexity to prevent mode collapse; and (4) a sandboxed code tool that mitigates catastrophic forgetting of intrinsic coding capabilities. Experiments on general tool-use benchmarks (BFCL, 𝜏2-Bench) and code agent tasks (RebenchT, CodeCI, BIRD) demonstrate a win-win outcome: TDScaling improves both tool-use generalization and inherent coding proficiency. Crucially, we show that trajectory diversity scaling attains a substantially higher performance ceiling than quantity scaling, establishing a resource-efficient paradigm for training robust code agents under data bottlenecks.