AutoGraphQL: A Bring-Your-Own-Schema Framework for Adapting Open-Weight LLMs to GraphQL APIs

Keya Battu, Parth Mehta, Harikrishnan Balagopal, Manish Kesarwani


Abstract
GraphQL has emerged as a widely adopted interface for accessing heterogeneous enterprise data. However, developing Natural Language to GraphQL (NL2GraphQL) systems remains challenging due to the need to rapidly adapt them to proprietary GraphQL schemas.

We present AutoGraphQL, a Bring-Your-Own-Schema framework for developing and evaluating NL2GraphQL systems. Given a GraphQL schema, AutoGraphQL synthesizes executable GraphQL queries, generates aligned natural language utterances, and validates them through schema and execution checks to automatically construct high-quality NL2GraphQL datasets. The framework additionally supports human-in-the-loop curation and an end-to-end workflow where the generated datasets are used to fine-tune and benchmark models.

Using AutoGraphQL, we construct a corpus of 17,621 NL2GraphQL instances spanning 173 GraphQL schemas and seven major query categories. Empirically, we show that fine-tuning open-weight models on AutoGraphQL-generated datasets substantially improves NL2GraphQL performance. Notably, the best fine-tuned 8B-parameter model achieves 42.57% benchmark accuracy, approaching Claude Sonnet 4.5 at 44.23%, while outperforming it on several GraphQL query categories. These results indicate that AutoGraphQL enables compact open-weight models to approach frontier-level performance on schema-grounded GraphQL generation. To support further research and benchmarking, we are releasing the training and test datasets under the MIT License. The dataset is available at https://nl2graphql-bench.github.io/.
Anthology ID:
2026.emnlp-industry.215
Volume:
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing: Industry Track
Month:
October
Year:
2026
Address:
Budapest, Hungary
Editors:
Pepa Atanasova, Gholamreza Haffari, Minghan Wang
Venue:
EMNLP
SIG:
SIGDAT
Publisher:
Association for Computational Linguistics
Note:
Pages:
3602–3613
Language:
URL:
https://aclanthology.org/2026.emnlp-industry.215/
DOI:
Bibkey:
Cite (ACL):
Keya Battu, Parth Mehta, Harikrishnan Balagopal, and Manish Kesarwani. 2026. AutoGraphQL: A Bring-Your-Own-Schema Framework for Adapting Open-Weight LLMs to GraphQL APIs. In Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing: Industry Track, pages 3602–3613, Budapest, Hungary. Association for Computational Linguistics.
Cite (Informal):
AutoGraphQL: A Bring-Your-Own-Schema Framework for Adapting Open-Weight LLMs to GraphQL APIs (Battu et al., EMNLP 2026)
Copy Citation:
PDF:
https://aclanthology.org/2026.emnlp-industry.215.pdf