Incivility and Rigidity: Evaluating the Risks of Fine-Tuning LLMs for Political Argumentation

Svetlana Churina, Kokil Jaidka


Abstract
Incivility on platforms such as Twitter (now X) and Reddit complicates the development of AI systems that can support productive, rhetorically sound political argumentation. We present experiments with GPT-3.5 Turbo fine-tuned on two contrasting datasets of political discourse: high-incivility Twitter replies to U.S. Congress and low-incivility posts from Reddit’s r/ChangeMyView. Our evaluation examines how data composition and prompting strategies affect the rhetorical framing and deliberative quality of model-generated arguments. Results show that Reddit-finetuned models generate safer but rhetorically rigid arguments, while cross-platform fine-tuning amplifies adversarial tone and toxicity. Prompt-based steering reduces overt toxicity (e.g., personal attacks) but cannot fully offset the influence of noisy training data. We introduce a rhetorical evaluation rubric—covering justification, reciprocity, alignment, and authority—and provide implementation guidelines for authoring, moderation, and deliberation-support systems.
Anthology ID:
2026.lrec-1.308
Volume:
Proceedings of the Fifteenth Language Resources and Evaluation Conference
Month:
May
Year:
2026
Address:
Palma de Mallorca, Spain
Editors:
Stelios Piperidis, Núria Bel, Henk van den Heuvel, Nancy Ide, Simon Krek, Antonio Toral
Venue:
LREC
SIG:
Publisher:
ELRA Language Resource Association
Note:
Pages:
3875–3883
Language:
External URL:
https://lrec.elra.info/lrec2026-main-308
DOI:
10.63317/5g48jjks5n35
Bibkey:
Cite (ACL):
Svetlana Churina and Kokil Jaidka. 2026. Incivility and Rigidity: Evaluating the Risks of Fine-Tuning LLMs for Political Argumentation. In Proceedings of the Fifteenth Language Resources and Evaluation Conference, pages 3875–3883, Palma de Mallorca, Spain. ELRA Language Resource Association.
Cite (Informal):
Incivility and Rigidity: Evaluating the Risks of Fine-Tuning LLMs for Political Argumentation (Churina & Jaidka, LREC 2026)
Copy Citation: