Gábor Madarász
Author directoryAlso published as: Gabor Madarasz
2026
Detecting Toxic Communication in Hungarian: Theory, Corpus, and Thread-Conditioned Classification
Péter Hatvani | Gábor Madarász
Proceedings of the 10th Workshop on Online Abuse and Harms (WOAH)
Péter Hatvani | Gábor Madarász
Proceedings of the 10th Workshop on Online Abuse and Harms (WOAH)
This paper addresses a conceptual and a resource deficit in Hungarian toxicity detection, which to date has been limited to message-level, single-label classification. Conceptually, we introduce an operational framework grounded in Culpeper’s impoliteness theory that distinguishes three tests: face-aggression against the interlocutor (T1), antisocial perlocutionary effect (T2), and community norm violation (T3). Because T2 is already operationalised by prior Hungarian work, we focus on T1 and T3, and we motivate why these are the legally and platform-compliance relevant dimensions. As a resource, we release a silver-annotated corpus of 148,474 conversational comments across 35,431 threads, with a 500-thread human gold subset carrying binary T1/T3 labels plus secondary pragmatic layers, following a thread-level conversational design. As a model, we introduce ModernBERT-HU-4k, a 149M-parameter Hungarian encoder pretrained with a 4,096-token context window and a purpose-built low-fertility tokenizer (1.52), fine-tuned for thread-conditioned per-comment classification and trained on a single consumer GPU. Against a benchmark of eleven zero-shot LLMs on the gold set, the fine-tuned encoder is competitive with the strongest frontier systems, and an ablation shows that thread context helps face-aggression most, precisely where the relational framing predicts. We also document that decomposing the construct yields norm-violation agreement comparable to prior single-label work while revealing face-aggression as a harder, context-dependent sub-problem.
2025
HuGME: A benchmark system for evaluating Hungarian generative LLMs
Noémi Ligeti-Nagy | Gabor Madarasz | Flora Foldesi | Mariann Lengyel | Matyas Osvath | Bence Sarossy | Kristof Varga | Győző Zijian Yang | Enikő Héja | Tamás Váradi | Gábor Prószéky
Proceedings of the Fourth Workshop on Generation, Evaluation and Metrics (GEM²)
Noémi Ligeti-Nagy | Gabor Madarasz | Flora Foldesi | Mariann Lengyel | Matyas Osvath | Bence Sarossy | Kristof Varga | Győző Zijian Yang | Enikő Héja | Tamás Váradi | Gábor Prószéky
Proceedings of the Fourth Workshop on Generation, Evaluation and Metrics (GEM²)
In this study, we introduce the Hungarian Generative Model Evaluation (HuGME) benchmark, a new framework designed to assess the linguistic proficiency of large language models (LLMs) in Hungarian. HuGME evaluates models across a diverse set of linguistic and reasoning skills, including bias, toxicity, faithfulness, relevance, summarization, prompt alignment, readability, spelling, grammaticality, and domain-specific knowledge through tasks like TruthfulQA and MMLU. We applied HuGME to a range of Hungarian LLMs, including those developed in-house as well as several publicly available models that claim Hungarian language proficiency. This paper presents the comparative results of these evaluations, shedding light on the capabilities of current LLMs in processing the Hungarian language. Through our analysis, we aim to both showcase the current state of Hungarian linguistic processing in LLMs and provide a foundational resource for future advancements in the field.