We describe the TUMHN team’s systems for the WMT26 Shared Tasks on LLMs with Lim- ited Resources for Slavic Languages. We partic- ipated in the Ukrainian track, covering machine translation, question answering, spell checking, grammar checking, and math reasoning tasks. We restructured our training dataset into a chat-style format and fine-tuned the Qwen3.5-2B model for the Ukrainian track in a multitask learn- ing setting. Our model outperforms the base- line on three of the five tasks, with significant improvements on Question Answering, Spell Checking, and Grammar Checking. In Machine Translation, the model shows a slight improve- ment for the English–Ukrainian direction. For Spell Checking and Grammar Checking specif- ically, the largest gains were observed in spell or grammatical error detection as opposed to correction. In Math Reasoning, however, our model does not outperform the baseline. The fine-tuned model 1 and training dataset 2 are publicly available on Hugging Face.

We analyze the generalization capabilities of a RoBERTa-based metaphor detection model on English verbs using the VU Amsterdam Metaphor Corpus. Since RoBERTa encoders are widely-used in metaphor detection, determining whether they rely on lexical memorization or learn transferable patterns is critical. We introduce a controlled lexical hold-out setup where all instances of selected target lemmas are strictly excluded from fine-tuning. We then compare model predictions on these Held-out lemmas against Exposed lemmas (verbs encountered during fine-tuning). While the model performs best on Exposed lemmas, it maintains robust performance on Held-out lemmas. This success does not correlate with verb lemma frequency, suggesting the generalization is likely not driven by frequency bias in pre-training. Further analysis reveals that sentence context alone is sufficient to match full-model performance on Held-out lemmas, whereas static verb-level embeddings are not. Geometrically, we find that the encoder projects Held-out instances into the same task-relevant subspaces as Exposed instances. Together, these results suggest that the model exhibits generalization capabilities, which appear to be driven by “learning the cue” (transferable contextual patterns) rather than “learning the word” (verb-specific memorization).

We explore the role of the visual modality and of vision transformers in predicting the compositionality of English noun compounds. Crucially, we contribute a framework to address the challenge of obtaining adequate images that represent non-compositional compounds (such as “couch potato”), making it relevant for any image-based approach targeting figurative language. Our method uses prompting strategies and diffusion models to generate images. Comparing and combining our approach with a state-of-the-art text-based approach reveals complementary contributions regarding features as well as degrees of abstractness in compounds.

Type- and token-based embedding architectures are still competing in lexical semantic change detection. The recent success of type-based models in SemEval-2020 Task 1 has raised the question why the success of token-based models on a variety of other NLP tasks does not translate to our field. We investigate the influence of a range of variables on clusterings of BERT vectors and show that its low performance is largely due to orthographic information on the target word, which is encoded even in the higher layers of BERT representations. By reducing the influence of orthography we considerably improve BERT’s performance.

Lexical semantic change detection is a new and innovative research field. The optimal fine-tuning of models including pre- and post-processing is largely unclear. We optimize existing models by (i) pre-training on large corpora and refining on diachronic target corpora tackling the notorious small data problem, and (ii) applying post-processing transformations that have been shown to improve performance on synchronic tasks. Our results provide a guide for the application and optimization of lexical semantic change detection models across various learning scenarios.

While there is a large amount of research in the field of Lexical Semantic Change Detection, only few approaches go beyond a standard benchmark evaluation of existing models. In this paper, we propose a shift of focus from change detection to change discovery, i.e., discovering novel word senses over time from the full corpus vocabulary. By heavily fine-tuning a type-based and a token-based approach on recently published German data, we demonstrate that both models can successfully be applied to discover new words undergoing meaning change. Furthermore, we provide an almost fully automated framework for both evaluation and discovery.