Teodora Stereciu
Author directory2026
Agent-Readiness of Synthesized API Tools: Parametric Gravity and the Limits of Documentation Grounding
Teodora Stereciu | Baohao Liao | Christian Herold | Shahram Khadivi | Christof Monz
Proceedings of the Second Workshop for REsearch on Agent Language Models (REALM 2026)
Teodora Stereciu | Baohao Liao | Christian Herold | Shahram Khadivi | Christof Monz
Proceedings of the Second Workshop for REsearch on Agent Language Models (REALM 2026)
We study whether LLM agents can autonomously synthesize deployment-ready MCP (Model Context Protocol) servers from raw REST API documentation, and whether the resulting servers are reliable enough for downstream agentic task completion. Evaluating 8 synthesis models across 14 APIs under three documentation conditions — full docs, no docs, and counterfactually mutated docs — we find that synthesis is broadly feasible: 11 of 14 APIs achieve sufficient tool coverage, and synthesized servers match or exceed human-authored references. The dominant mediating factor is not model capability but parametric gravity: for well-known APIs (GitHub, Stripe, Shopify), training-time knowledge overrides in-context documentation, rendering documentation provision neutral or harmful. A counterfactual mutation study confirms this: high-familiarity APIs show near-zero adherence to renamed parameters, while low-familiarity APIs show near-complete adherence. Logit-lens probing shows the representations of memory and context become increasingly separable in the final transformer layers, consistent with deep feed-forward MLP modules as the site of parametric association storage. Synthesis coverage (SYNTH) correlates strongly with agent task completion (r = 0.937), providing a reliable pre-flight quality signal without requiring agent evaluation.