Parth Mehta
Author directory2026
AutoGraphQL: A Bring-Your-Own-Schema Framework for Adapting Open-Weight LLMs to GraphQL APIs
Keya Battu | Parth Mehta | Harikrishnan Balagopal | Manish Kesarwani
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing: Industry Track
Keya Battu | Parth Mehta | Harikrishnan Balagopal | Manish Kesarwani
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing: Industry Track
GraphQL has emerged as a widely adopted interface for accessing heterogeneous enterprise data. However, developing Natural Language to GraphQL (NL2GraphQL) systems remains challenging due to the need to rapidly adapt them to proprietary GraphQL schemas.
We present AutoGraphQL, a Bring-Your-Own-Schema framework for developing and evaluating NL2GraphQL systems. Given a GraphQL schema, AutoGraphQL synthesizes executable GraphQL queries, generates aligned natural language utterances, and validates them through schema and execution checks to automatically construct high-quality NL2GraphQL datasets. The framework additionally supports human-in-the-loop curation and an end-to-end workflow where the generated datasets are used to fine-tune and benchmark models.
Using AutoGraphQL, we construct a corpus of 17,621 NL2GraphQL instances spanning 173 GraphQL schemas and seven major query categories. Empirically, we show that fine-tuning open-weight models on AutoGraphQL-generated datasets substantially improves NL2GraphQL performance. Notably, the best fine-tuned 8B-parameter model achieves 42.57% benchmark accuracy, approaching Claude Sonnet 4.5 at 44.23%, while outperforming it on several GraphQL query categories. These results indicate that AutoGraphQL enables compact open-weight models to approach frontier-level performance on schema-grounded GraphQL generation. To support further research and benchmarking, we are releasing the training and test datasets under the MIT License. The dataset is available at https://nl2graphql-bench.github.io/.
We present AutoGraphQL, a Bring-Your-Own-Schema framework for developing and evaluating NL2GraphQL systems. Given a GraphQL schema, AutoGraphQL synthesizes executable GraphQL queries, generates aligned natural language utterances, and validates them through schema and execution checks to automatically construct high-quality NL2GraphQL datasets. The framework additionally supports human-in-the-loop curation and an end-to-end workflow where the generated datasets are used to fine-tune and benchmark models.
Using AutoGraphQL, we construct a corpus of 17,621 NL2GraphQL instances spanning 173 GraphQL schemas and seven major query categories. Empirically, we show that fine-tuning open-weight models on AutoGraphQL-generated datasets substantially improves NL2GraphQL performance. Notably, the best fine-tuned 8B-parameter model achieves 42.57% benchmark accuracy, approaching Claude Sonnet 4.5 at 44.23%, while outperforming it on several GraphQL query categories. These results indicate that AutoGraphQL enables compact open-weight models to approach frontier-level performance on schema-grounded GraphQL generation. To support further research and benchmarking, we are releasing the training and test datasets under the MIT License. The dataset is available at https://nl2graphql-bench.github.io/.
2025
PNLP at MEDIQA-OE 2025: A Zero-Shot Prompting Strategy with Gemini for Medical Order Extraction
Parth Mehta
Proceedings of the 7th Clinical Natural Language Processing Workshop
Parth Mehta
Proceedings of the 7th Clinical Natural Language Processing Workshop
Medical order extraction from doctor-patient conversations presents a critical challenge in reducing clinical documentation burden and ensuring accurate capture of patient care instructions. This paper describes our system for the MEDIQA-OE 2025 shared task using the ACI-Bench and PriMock57 datasets, which achieved second place on the public leaderboard with an average score of 0.6014 across four metrics: description ROUGE-1 F1, reason ROUGE-1 F1, order-type strict F1, and provenance multi-label F1. Unlike traditional approaches that rely on fine-tuned biomedical language models, we demonstrate that a carefully engineered zero-shot prompting strategy using Gemini 2.5 Pro can achieve competitive performance without requiring model training or GPU resources. Our approach employs a deterministic state-machine prompt design incorporating chain-of-thought reasoning, self-verification protocols, and structured JSON output generation. The system particularly excels in reason extraction, achieving 0.4130 ROUGE-1 F1, the highest among the top performing teams. Our results suggest that advanced prompt engineering can effectively bridge the gap between general-purpose large language models and specialized clinical NLP tasks, offering a computationally efficient and immediately deployable alternative to traditional fine-tuning approaches with significant implications for resource-constrained healthcare settings.
2016
From Extractive to Abstractive Summarization: A Journey
Parth Mehta
Proceedings of the ACL 2016 Student Research Workshop
Parth Mehta
Proceedings of the ACL 2016 Student Research Workshop