We present a forensic analysis of the generation pipeline behind a recent AI driven influence campaign. We introduce PROPAGIA, a corpus of 2,646 propagandist French articles from the Storm-1516/CopyCop campaign disclosed by VIGINUM and INSIKT GROUP in 2025. For comparison, we rely on SIPA, a corpus of human-written French mainstream press from the same period. Using topic modeling, vagueness and sentiment analysis, we first isolate persuasion techniques characteristic of propaganda, with PROPAGIA far exceeding SIPA in vagueness, subjectivity and negativity, and citing fewer sources. We then find prompt instruction leaks on 50 of the 84 PROPAGIA websites, including a verbatim ten-point editorial specification accounting for several of these differences, together with high cross-article redundancy. Finally, we show that rewriting-based detection supports INSIKT GROUP’s attribution to the Llama 3 family, but also suggests the involvement of Mistral-family models.

Large language models (LLMs) can convincingly imitate human writing styles, yet it remains unclear how much stylistic information is encoded in embeddings from any language model and retained after LLM rewriting. We investigate these questions in French, using a controlled literary dataset to quantify the effect of stylistic variation via changes in embedding dispersion. We observe that embeddings reliably capture authorial stylistic features and that these signals persist after rewriting, while also exhibiting LLM-specific patterns. These analytical results offer promising directions for authorship imitation detection in the era of language models.

Cet article vise à étudier comment le style d’écriture influence la dispersion des plongements vectoriels de divers grands modèles de langage. Alors que les premiers modèles de type transformeur étaient prin- cipalement axés sur la modélisation thématique, cette étude examine le rôle du style d’écriture dans la configuration de l’espace vectoriel. À partir d’un corpus littéraire faisant varier thématiques et styles, nous comparons la sensibilité des modèles de langage en français et en anglais. En analysant ainsi l’impact spécifique du style sur la dispersion vectorielle, nous cherchons à mieux comprendre com- ment les modèles de langage traitent l’information stylistique, contribuant ainsi à leur interprétabilité globale. Ceci est un résumé de l’article “Embedding Style Beyond Topics: Analyzing Dispersion Effects Across Different Language Models” publié dans les actes de la conférence COLING 2025 (Icard et al., 2025) et accessible à l’URL : https://aclanthology.org/2025.coling-main.236/.

This paper analyzes how writing style affects the dispersion of embedding vectors across multiple, state-of-the-art language models. While early transformer models primarily aligned with topic modeling, this study examines the role of writing style in shaping embedding spaces. Using a literary corpus that alternates between topics and styles, we compare the sensitivity of language models across French and English. By analyzing the particular impact of style on embedding dispersion, we aim to better understand how language models process stylistic information, contributing to their overall interpretability.