This paper describes our submission to the WMT26 General Machine Translation shared task for the English → Belarusian direction. In lower-resourced morphologically rich languages like Belarusian, open-weight large language models (LLMs) frequently struggle with orthographic fidelity, hallucinated vocabulary, and negative transfer from related high-resource languages. To address these issues, we implemented Krosny, an iterative, resource-augmented pipeline built on top of TranslateGemma 12B. The system translates each segment in three stages: (1) dictionary-augmented generation, (2) refinement using a reference grammatical database, and (3) rule-based post-processing followed by an LLM-as-a-judge candidate selection. We find a small but consistent improvement over the baseline TranslateGemma 12B. In comparison with other constrained-track systems of WMT26, the performance of Krosny is mid-tier. We release the source code of our implementation (https://github.com/volchek/Krosny-WMT26).

We explore the use of large language models (LLMs) for morphological disambiguation in Belarusian, a low-resource language. The pipeline has two stages: a rule-based analyzer generates candidate lemmas and grammatical tags, which an LLM then disambiguates in context. Initial evaluation of ChatGPT, Claude, and Gemini on a gold-standard sample shows high accuracy. We scale this approach to a 375K-word corpus using Gemini and compare the results against a neural baseline (Stanza). Manual review of discrepancies suggests that the LLM-based approach outperforms the baseline, offering a solution for corpus annotation in Belarusian.

As large language models saturate existing benchmarks, language- and culture-specific tasks remain a productive source of problems that are challenging for current models yet solvable by humans. We present SQARB (Sophisticated Question Answering and Reasoning in Belarusian), the first dedicated, natively authored question answering resource for Belarusian, a low-resource East Slavic language. SQARB contains 500 short-answer questions manually written by domain experts, based on doctoral dissertations defended in Belarus between 1994 and 2023, covering five topics: the Belarusian language, literature, folklore, art, and history. The questions are designed to be culture-specific, hard to retrieve online, evergreen, and to require a mix of factual knowledge and reasoning. The dataset is split into an easier development set and a harder test set, both filtered adversarially against locally deployed models. We evaluate near-frontier models without tool use on SQARB and find a large gap between proprietary and open-weight models, with the best-performing model (Gemini 3.1 Pro) achieving 42.3% accuracy on the test set. Models consistently exhibit overconfidence and perform relatively better on questions requiring plausible reasoning, rather than pure factual recall. We release SQARB under gated access with a canary string.1

We examine how the capabilities of large language models (LLMs) have evolved on eight Belarusian language tasks contributed in 2023 to OpenAI’s Evals framework. We evaluate state-of-the-art models both on the original development sets and newly created test sets. Results demonstrate significant but non-uniform progress over this period: some tasks are almost saturated, while others show minor improvement beyond trivial baselines. Error analysis shows that certain challenges haven’t yet been addressed, e.g. misidentification of non-words as legitimate vocabulary, or conversion from modern to classical orthography. We release the datasets and the generated completions (https://doi.org/10.5281/zenodo.18163825).

In the epoch of multilingual large language models (LLMs), it is still challenging to evaluate the models’ understanding of lower-resourced languages, which motivates further development of expert-crafted natural language understanding benchmarks. We introduce BelarusianGLUE — a natural language understanding benchmark for Belarusian, an East Slavic language, with ≈15K instances in five tasks: sentiment analysis, linguistic acceptability, word in context, Winograd schema challenge, textual entailment. A systematic evaluation of BERT models and LLMs against this novel benchmark reveals that both types of models approach human-level performance on easier tasks, such as sentiment analysis, but there is a significant gap in performance between machine and human on a harder task — Winograd schema challenge. We find the optimal choice of model type to be task-specific: e.g. BERT models underperform on textual entailment task but are competitive for linguistic acceptability. We release the datasets (https://hf.co/datasets/maaxap/BelarusianGLUE) and evaluation code (https://github.com/maaxap/BelarusianGLUE).