For many languages, grammatical error correction (GEC) remains underexplored due to the lack of dedicated datasets and evaluation benchmarks. Moreover, many existing GEC datasets are based on texts written by second-language learners, which narrows the research focus to specific error types. To address these gaps, we introduce a new Polish GEC benchmark, AMU-GEC, built from texts written by native Polish speakers and manually annotated by expert linguists. The benchmark is released as a machine learning challenge, without revealing the expected labels, to prevent contamination. To set baselines, we evaluate recent multilingual and Polish LLMs, finding that smaller models may perform better when trained on monolingual (Polish) data. We also conduct a human analysis of the best performing model to gain more insights about the model behavior when it is different from human annotations. The analysis shows that most of the changes suggested by the model do not introduce errors into the texts; instead, they typically either slightly improve text fluency or represent valid alternative corrections.