Yeqing Liu

Author directory

2026

Corpus Query Language (CQL) supports precise corpus analysis but requires expertise in annotations, regular expressions, structural scopes, and platform-specific syntax. Natural-language interfaces simplify access, but their answers can be difficult to inspect, reproduce, or revise. We study Text-to-CQL as executable semantic parsing, mapping intents to inspectable queries. We introduce ECQL, a Sketch-Engine-oriented benchmark of 3,572 natural language–CQL pairs comprising 811 tutorial-derived and 2,761 synthetic examples, with a default split of 3,215 training and 357 test examples. ECQL focuses on Sketch Engine normalization, CQL Diagnostic Profile (CQL-DP) annotations, controlled semantic contrasts, audit-oriented metadata, and generalization protocols, alongside an evaluation suite covering syntactic validity, exact match, canonical match, and execution match. The full 357-example headline test set is independently double-annotated and adjudicated, with 96.9% agreement for intent clarity, 94.7% agreement for CQL–intent alignment, and a 3.1% post-adjudication intent-match error rate. Experiments with prompting, retrieval-augmented prompting, syntax repair, DeepSeek-R1 reasoning prompting, a GPT-4o Agent/API alternative, and supervised fine-tuning reveal persistent difficulties with corpus-query semantics. The best fine-tuned model reaches 85.6% execution accuracy on the random split, but drops to 68.9% on template-disjoint evaluation. ECQL provides a reproducible testbed for auditable semantic parsing over specialized linguistic query languages.