For many languages, grammatical error correction (GEC) remains underexplored due to the lack of dedicated datasets and evaluation benchmarks. Moreover, many existing GEC datasets are based on texts written by second-language learners, which narrows the research focus to specific error types. To address these gaps, we introduce a new Polish GEC benchmark, AMU-GEC, built from texts written by native Polish speakers and manually annotated by expert linguists. The benchmark is released as a machine learning challenge, without revealing the expected labels, to prevent contamination. To set baselines, we evaluate recent multilingual and Polish LLMs, finding that smaller models may perform better when trained on monolingual (Polish) data. We also conduct a human analysis of the best performing model to gain more insights about the model behavior when it is different from human annotations. The analysis shows that most of the changes suggested by the model do not introduce errors into the texts; instead, they typically either slightly improve text fluency or represent valid alternative corrections.

PolEval is an annual shared-task evaluation campaign dedicated to advancing natural language processing for the Polish language. This paper presents an overview of PolEval 2025, the eighth edition of the campaign, which included three completed tasks covering machine-generated text detection, gender-inclusive language generation, and speech emotion recognition. The evaluation was conducted using standardised datasets and metrics via the AmuEval platform. PolEval 2025 attracted 15 teams and over 100 submissions, demonstrating continued engagement from the Polish NLP community. We describe the organisation of the campaign, the evaluation setup, and the role of PolEval in fostering reproducible research and community-driven benchmarking.

We present a new method to detect anomalies in texts (in general: in sequences of any data), using language models, in a totally unsupervised manner. The method considers probabilities (likelihoods) generated by a language model, but instead of focusing on low-likelihood tokens, it considers a new metric defined in this paper: oddballness. Oddballness measures how “strange” a given token is according to the language model. We demonstrate in grammatical error detection tasks (a specific case of text anomaly detection) that oddballness is better than just considering low-likelihood events, if a totally unsupervised setup is assumed.

Decoder-only large language models have shown superior performance in the fluency-edit English Grammatical Error Correction, but their adaptation for minimal-edit English GEC is still underexplored. To improve their effectiveness in the minimal-edit approach, we explore the error rate adaptation topic and propose a novel training schedule method. Our experiments set a new state-of-the-art result for a single-model system on the BEA-test set. We also detokenize the most common English GEC datasets to match the natural way of writing text. During the process, we find that there are errors in them. Our experiments analyze whether training on detokenized datasets impacts the results and measure the impact of the usage of the datasets with corrected erroneous examples. To facilitate reproducibility, we have released the source code used to train our models.