Mingqian Zhang

Author directory

2026

Model-internal parallelization improves generation quality by introducing multiple parallel computation paths, but its additional latency and computational overhead hinder practical deployment. Recent studies reveal a functional division across model depth: shallow layers primarily encode input-conditioned features, whereas deeper layers are more involved in reasoning and generation. Since parallel paths process the same input and are optimized toward the same generation objective, their early representations are likely to overlap substantially, suggesting that early-layer parallelism may induce redundant computation. Motivated by this observation, we propose Adaptive Depth Parallelism (ADP), a framework that shares computation in shallow layers and activates parallel branches only after an adaptively selected bifurcation layer. We provide a sufficient-condition analysis for delayed branching and identify an empirical transition in inter-path cosine similarity across depth. This motivates using the stable minimum-similarity layer as a practical estimate of the bifurcation point. Experiments show that ADP achieves 104.83% of baseline performance while using only 39.4% of the computational load.