We present a comprehensive study of LLMs for argument mining, comparing four learning paradigms against a strong encoder-only baseline across three heterogeneous corpora to assess cross-domain generalisation. We evaluate in-context learning, supervised fine-tuning (SFT), and cold-start reinforcement fine-tuning (RFT) via GRPO. Our results demonstrate that BERT-style encoder-only transformers consistently outperform much larger decoder-only LLMs across all settings. Further mechanistic analysis reveals that the central bottleneck is not knowledge but expression: linear probing of frozen hidden states recovers argument relations at 69–80% accuracy from base models, yet edge generation remains poor across all paradigms. Attention analysis confirms a universal structural blindness across 27 tested conditions.

This article examines LLMs’ ability to correctly label simple inferences with partisan conclusions. For this, we develop a dataset with both formal and material inferences, containing logically equivalent pairs of inferences with conclusions that favor either the political left or the political right. This allows us to focus on political bias as a source of decrease in performance. Our samples are synthetically generated and thus highly controlled, covering both English and German. We assess the performance of 16 configurations of both open and proprietary state-of-the-art LLMs on that dataset, finding generally unreliable performance as well as widespread political bias which, in the case of the English samples, persists throughout our experimental settings.