Don’t Count the Edits, Judge by the Outcome Alone: Reward-Based Evaluation for Grammatical Error Correction

Hayeong Ryu, Sunhee Jo, Seunguk Yu, YoungBin Kim


Abstract
Grammatical error correction (GEC) evaluation has traditionally relied on reference or edit overlap, which can penalize valid rewrites that differ from gold corrections. Reference-free metrics reduce this dependence, but evaluating whether a fluent output is a valid correction of the source remains challenging. We propose SURE, a source-conditioned reward evaluator trained on within-source preferences spanning minimal-edit and rewrite-oriented corrections. SURE jointly learns an overall reward with criteria-level supervision for grammaticality, faithfulness, and fluency, together with span-level grounding for source-side error resolution. Experiments on SEEDA show that SURE performs competitively against strong baselines, with particular gains on rewrite-style corrections and more disentangled criteria-level diagnostics. Our code is available at https://github.com/hayeonggg/SURE.
Anthology ID:
2026.findings-emnlp.311
Volume:
Findings of the Association for Computational Linguistics: EMNLP 2026
Month:
October
Year:
2026
Address:
Budapest, Hungary
Editors:
Hung-Yi Lee, Sunipa Dev, Jessy Li, Desmond Elliott
Venue:
Findings
SIG:
SIGDAT
Publisher:
Association for Computational Linguistics
Note:
Pages:
6581–6597
Language:
URL:
https://aclanthology.org/2026.findings-emnlp.311/
DOI:
Bibkey:
Cite (ACL):
Hayeong Ryu, Sunhee Jo, Seunguk Yu, and YoungBin Kim. 2026. Don’t Count the Edits, Judge by the Outcome Alone: Reward-Based Evaluation for Grammatical Error Correction. In Findings of the Association for Computational Linguistics: EMNLP 2026, pages 6581–6597, Budapest, Hungary. Association for Computational Linguistics.
Cite (Informal):
Don’t Count the Edits, Judge by the Outcome Alone: Reward-Based Evaluation for Grammatical Error Correction (Ryu et al., Findings 2026)
Copy Citation:
PDF:
https://aclanthology.org/2026.findings-emnlp.311.pdf