Shaofeng Liang

Author directory

2026

Large language models (LLMs) have undergone rapid growth, forming a heterogeneous ecosystem whose model relationships and capability profiles lack comprehensive documentation. Organizing such an ecosystem requires reusable model representations that are comparable across architectures while still reflecting how each model processes shared inputs. We propose ABLE (Attribution-Based Large-model Embedding), a training-free framework that represents each model in an attribution space. Given a shared probing corpus, ABLE computes feature attributions, aligns token-level attribution scores into a tokenizer-agnostic word-level representation, and projects the resulting attribution patterns into compact model embeddings. These embeddings summarize model-level input-dependence patterns and can be reused across downstream tasks. We construct ABLE representations for 239 LLMs and evaluate them on model organization and relationship analysis, model routing, and benchmark score prediction. Across these tasks, ABLE captures meaningful model-level structure, achieves stronger relationship prediction than parameter- and output-based baselines, supports competitive model routing, and predicts benchmark scores from frozen representations, while theoretical analysis further supports the stability of the representation.