Alba Pérez-Montero
Author directoryAlso published as: Alba Perez-Montero
2026
How Noisy is Automatic Annotation? Characterizing Label Noise from Generative Models in Multi-Dimensional Spanish Subjectivity Annotation
Alba Pérez-Montero | Elena Lloret | Paloma Moreda Pozo
Proceedings of the 11th Workshop on Natural User-generated Text
Alba Pérez-Montero | Elena Lloret | Paloma Moreda Pozo
Proceedings of the 11th Workshop on Natural User-generated Text
Although human variability has traditionally been the primary source of annotation noise, the increasing use of Large Language Models (LLMs) for automated labeling is shifting this landscape toward systematic, machine-generated label noise. This challenge is particularly severe in Subjectivity Detection (SD), where the behavior of automated errors remains largely unexamined. This study characterizes the extent and linguistic nature of automated annotation noise in Spanish SD. Errors are evaluated as omissions (under-generation) or additions (over-generation) across six subjectivity dimensions: Doubt, Certainty, Specificity, Non-specificity, Distancing, and Participation. Using an expert-annotated human gold standard as a reference, a traditional machine learning control is benchmarked against three state-of-the-art LLM families (GPT, Gemini, and Llama). The results reveal a clear architectural divide: while the traditional ML framework predominantly generates omission errors (63.99%), only detecting markers it was explicitly programmed to recognize, LLMs like Gemini and Llama lean heavily toward addition errors (exceeding 51%), guessing and hallucinating markers instead of leaving uncertain cases unlabeled. Although Gemini minimizes overall noise, achieving F1-scores up to 0.96 in specific dimensions, Certainty remains highly susceptible to errors, showing a 53.96% error rate in the traditional ML control. Ultimately, this diagnostic evaluation of LLMs as annotators in this specific subjective task reveals that automated systems introduce systematic label noise, demonstrating that human-in-the-loop participation is indispensable to resolve these errors and ensure reliable dataset curation.
Pragmatic Profiling for Disinformation Detection: An Exploratory Analysis of Stylistic Features in Spanish News
Alba Perez-Montero | María Miró Maestre | Elena Lloret | Paloma Moreda
Proceedings of the Second International Conference on Natural Language Processing and Artificial Intelligence for Cyber Security
Alba Perez-Montero | María Miró Maestre | Elena Lloret | Paloma Moreda
Proceedings of the Second International Conference on Natural Language Processing and Artificial Intelligence for Cyber Security
The spread of misleading and fabricated information has made automatic disinformation detection a central challenge for Natural Language Processing. While most approaches have focused on lexical, syntactic, or semantic cues, deceptive discourse is also shaped by pragmatic choices that reflect how information is framed, qualified, and directed toward readers. This paper investigates whether pragmatic information can improve disinformation detection in Spanish by explicitly incorporating two annotation layers: communicative intentions and subjectivity markers. We compare the impact of these pragmatic features across three modeling paradigms: traditional classifiers, an encoder-based transformer (RoBERTa), and generative language models (GPT-oss and Mistral-small). Our results show that pragmatic augmentation consistently improves over text-only baselines, with subjectivity markers displaying stronger discriminative power than intention labels. Statistical testing further confirms that the observed gains are robust for the generative models evaluated. These findings support the view that authorial stance and communicative purpose provide useful complementary evidence for veracity classification.
A Multilingual Linguistic Analysis of Human vs LLM-Generated News in a Disinformation Context
Silvia Gargova | Alba Perez-Montero | Elena Lloret Pastor | Paloma Moreda Pozo
Proceedings of the 1st Workshop on Information Disorder (InDor) @ LREC 2026
Silvia Gargova | Alba Perez-Montero | Elena Lloret Pastor | Paloma Moreda Pozo
Proceedings of the 1st Workshop on Information Disorder (InDor) @ LREC 2026
The rise of Large Language Models has shifted the Information Disorder landscape toward automated threats. This study investigates the linguistic construction of synthetic news by comparing GPT-5, Gemini 2.5, and Grok 4 across English, Spanish, and Bulgarian. Using multilingual human-authored verified news and disinformation as seeds, we analyze how prompt informativeness and model architecture influence deceptive content production. Our methodology employs five metrics: semantic similarity, factual consistency, readability, lexical richness, and persuasion technique frequency. Our analysis reveals that while prompt scarcity leads to informational loss, LLMs maintain a homogenized stylistic template regardless of input length. Unlike human authors, who intensify rhetorical and emotional markers to drive deceptive intent, LLMs adhere to a neutral register. This study identifies distinct statistical patterns in generated content characterized by hyper-standardized readability and high lexical density (p < 0.001). These features serve as robust “LLM signatures”, enabling a classification accuracy of 96% across English, Spanish, and Bulgarian. These findings suggest that generated disinformation relies on invariant syntactic structures rather than nuanced human rhetoric, providing a framework for detection tools centered on structural patterns rather than content veracity.
2025
Detecting Deception in Disinformation across Languages: The Role of Linguistic Markers
Alba Perez-Montero | Silvia Gargova | Elena Lloret | Paloma Moreda
Proceedings of the 15th International Conference on Recent Advances in Natural Language Processing - Natural Language Processing in the Generative AI Era
Alba Perez-Montero | Silvia Gargova | Elena Lloret | Paloma Moreda
Proceedings of the 15th International Conference on Recent Advances in Natural Language Processing - Natural Language Processing in the Generative AI Era
The unstoppable proliferation of news driven by the rise of digital media has intensified the challenge of news verification. Natural Language Processing (NLP) offers solutions, primarily through content and context analysis. Recognizing the vital role of linguistic analysis, this paper presents a multilingual study of linguistic markers for automated deceptive fake news detection across English, Spanish, and Bulgarian. We compiled datasets in these languages to extract and analyze both general and specific linguistic markers. We then performed feature selection using the SelectKBest algorithm, applying it to various classification models with different combinations of general and specific linguistic markers. The results show that Logistic Regression and Support Vector Machine classification models achieved F1-scores above 0.8 for English and Spanish. For Bulgarian, Random Forest yielded the best results with an F1-score of 0.73. While these markers demonstrate potential for transferability to other languages, results may vary due to inherent linguistic characteristics. This necessitates further experimentation, especially in low-resource languages like Bulgarian. These findings highlight the significant potential of our dataset and linguistic markers for multilingual deceptive news detection.