How Far Can Prompting Go for Minimal-Edit Ukrainian Grammatical Error Correction?

Kateryna Karpo, Artem Chernodub


Abstract
Fine-tuned Large Language Models (LLMs) dominate in Ukrainian grammatical error correction (GEC), while API-accessed LLMs remain nearly untested on minimal-edit benchmarks. We evaluate 11 commercial LLMs from four providers and one open-source Ukrainian model on the UNLP 2023 GEC-only benchmark, comparing zero-shot, few-shot, minimal-edits, and LLM-assisted prompt optimization strategies. Our best configuration (Gemini 3.1-Pro) reaches F0.5=69.22, closing over 90% of the gap to fine-tuned SOTA (F0.5=73.14). For zero-shot prompts, only Claude models benefit from Ukrainian instructions. However, the best overall results for all models use Ukrainian minimal-edits prompts, whose language-specific rules require Ukrainian to express precisely. LLM-assisted prompt optimization on top of minimal-edits + few-shot achieves the highest score. Detailed minimal-edits instructions yield the largest gains for punctuation and case errors but cause the model to abandon several low-frequency categories. Delving into error analysis, we identify five recurring overcorrection patterns tied to Ukrainian-specific linguistic phenomena. Code, prompts, and outputs are publicly available.
Anthology ID:
2026.unlp-1.13
Volume:
Proceedings of the Fifth Ukrainian Natural Language Processing Conference (UNLP 2026)
Month:
May
Year:
2026
Address:
Lviv, Ukraine
Editor:
Mariana Romanyshyn
Venue:
UNLP
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
136–154
Language:
URL:
https://aclanthology.org/2026.unlp-1.13/
DOI:
Bibkey:
Cite (ACL):
Kateryna Karpo and Artem Chernodub. 2026. How Far Can Prompting Go for Minimal-Edit Ukrainian Grammatical Error Correction?. In Proceedings of the Fifth Ukrainian Natural Language Processing Conference (UNLP 2026), pages 136–154, Lviv, Ukraine. Association for Computational Linguistics.
Cite (Informal):
How Far Can Prompting Go for Minimal-Edit Ukrainian Grammatical Error Correction? (Karpo & Chernodub, UNLP 2026)
Copy Citation:
PDF:
https://aclanthology.org/2026.unlp-1.13.pdf