Proceedings of the Second International Conference on Natural Language Processing and Artificial Intelligence for Cyber Security
Ruslan Mitkov, Rafael Muñoz, Elena Lloret, Tharindu Ranasinghe, Ernesto L. Estevanell-Valladares, Salima Lamsiyah, Andrés Montoyo, Saad Ezzini (Editors)
- Anthology ID:
- 2026.nlpaics-1
- Month:
- June
- Year:
- 2026
- Address:
- Alicante, Spain
- Venue:
- NLPAICS
- Event:
- International Conference on Natural Language Processing and Artificial Intelligence for Cyber Security (2026)
- SIG:
- Publisher:
- Department of Languages and Information Systems, University of Alicante
- URL:
- https://aclanthology.org/2026.nlpaics-1/
- DOI:
- PDF:
- https://aclanthology.org/2026.nlpaics-1.pdf
Proceedings of the Second International Conference on Natural Language Processing and Artificial Intelligence for Cyber Security
Ruslan Mitkov | Rafael Muñoz | Elena Lloret | Tharindu Ranasinghe | Ernesto L. Estevanell-Valladares | Salima Lamsiyah | Andrés Montoyo | Saad Ezzini
Ruslan Mitkov | Rafael Muñoz | Elena Lloret | Tharindu Ranasinghe | Ernesto L. Estevanell-Valladares | Salima Lamsiyah | Andrés Montoyo | Saad Ezzini
Exploring Cross-Lingual Transfer in Transformer-Based Fraud Detection Models
Ivan Martinez-Murillo | Robiert Sepúlveda-Torres | Juan Pablo Consuegra-Ayala
Ivan Martinez-Murillo | Robiert Sepúlveda-Torres | Juan Pablo Consuegra-Ayala
Digital communication channels have become major vectors for large-scale cyber fraud, including spam, phishing, and recruitment scams, causing significant financial losses and eroding user trust. While transformer-based models have improved text classification, most systems remain English-centric, limiting their effectiveness in multilingual environments and across diverse fraud typologies. In this work, we hypothesize that fraud detection models trained in a single language can exhibit cross-lingual generalization capabilities. We investigate the use of Natural Language Processing (NLP) for detecting multiple forms of textual fraud by developing and evaluating three transformer-based discriminative models targeting spam, phishing, and fake job postings. Specifically, we compare two architectures—MrBERT (308M parameters) and mROBERTa (283M parameters)—across English, Spanish, and Valencian data. Our results demonstrate that while models trained exclusively on English achieve near-perfect performance in-language (F1 ≈ 0.98), they exhibit limited cross-lingual generalization in other languages such as Spanish (F1 ≈ 0.44 for phishing detection, and F1 ≈ 0.85 for spam detection). However, incorporating multilingual training data dramatically improves performance in target languages (F1 ≈ 0.95–0.99) while maintaining or even enhancing English accuracy. We further observe that cross-lingual transfer is most effective when datasets are parallel or closely aligned. These findings underscore the critical importance of multilingual data inclusion for building robust, scalable cybersecurity frameworks in diverse linguistic settings.
Since the early 2010s, internet nationalists have been active on Japanese forums, blogs, and social networking services, spreading hate speech and conspiracy. At the same, the involuntary celibate community has gained international notoriety due to acts of mass violence committed by self-identified members. Despite several instances of misogyny motivated violence in Japan, there is little known about Japanese involuntary celibate culture and identity. This paper introduces Grieve-JA, a psycholinguistic dictionary for the analysis of violence, threat, and grievance in Japanese text. The paper outlines the process of developing the dictionary and evaluating it across different corpora representing both neutral and grievance-filled language. As a final measure, the dictionary is used to compare comments from two forums on the Japanese anonymous message board 5channel with comments from Bluesky. Results indicate that Grieve-JA is a useful tool for analyzing extreme language in Japanese as a complement to other psycholinguistic measures.
Pragmatic Profiling for Disinformation Detection: An Exploratory Analysis of Stylistic Features in Spanish News
Alba Perez-Montero | María Miró Maestre | Elena Lloret | Paloma Moreda
Alba Perez-Montero | María Miró Maestre | Elena Lloret | Paloma Moreda
The spread of misleading and fabricated information has made automatic disinformation detection a central challenge for Natural Language Processing. While most approaches have focused on lexical, syntactic, or semantic cues, deceptive discourse is also shaped by pragmatic choices that reflect how information is framed, qualified, and directed toward readers. This paper investigates whether pragmatic information can improve disinformation detection in Spanish by explicitly incorporating two annotation layers: communicative intentions and subjectivity markers. We compare the impact of these pragmatic features across three modeling paradigms: traditional classifiers, an encoder-based transformer (RoBERTa), and generative language models (GPT-oss and Mistral-small). Our results show that pragmatic augmentation consistently improves over text-only baselines, with subjectivity markers displaying stronger discriminative power than intention labels. Statistical testing further confirms that the observed gains are robust for the generative models evaluated. These findings support the view that authorial stance and communicative purpose provide useful complementary evidence for veracity classification.
Transformer-Assisted LLM-Based Source Code Summarisation: to Enable More Secure Software Development
Jesse Phillips | Tracy Hall | Paul Rayson | Mo El-Haj
Jesse Phillips | Tracy Hall | Paul Rayson | Mo El-Haj
Neural Source Code Summarisation (NSCS) aims to generate natural language summaries of source code to improve developer and maintainer understanding of code. Source code summaries are vital for the maintenance phase of the Secure Software Development Lifecycle (SSDLC) as they improve maintainers’ understanding of code, in order to reduce the number of bugs and vulnerabilities in a software system. However, summaries are often missing, incomplete, or outdated in many software systems. Solutions to this problem use small, task-specific Transformer models or code-aware Large Language Models (LLMs). Task-specific Transformer-generated summaries often score well across many NLG metrics but these NLG metrics reward lexical overlap, rather than summary quality. Conversely, LLMs’ ability to capture semantics in order to produce high-quality summaries presents an exciting solution to this problem, especially with the increased availability of LLMs and the increase in capability of workstation hardware over recent years meaning that some LLMs can be run from developers’ workstations. However, LLM summaries of code often differ greatly from developer-written summaries in terms of the words and phrases used due to the abstractive nature of LLMs, resulting in low scores across NLG metrics. We show how combining these two methods by using Transformer-generated summaries in prompt engineering may enable LLMs to create better source code summaries in order to better enable software practitioners to maintain secure systems. We prompt four LLMs, using four different prompts - with the use of a task-specific Transformer to aid the LLMs in the prompts. We present "Transformer-Assisted LLM-Based Source Code Summarisation" - a method through which, we observe an improvement of 7.8% BLEU-4 and 5% BERTScore on CodeLlama.
Code Without Context: Can We Trust LLMs to Test Software from Informal Descriptions?
Amneh Al Abdi | Saad Ezzini
Amneh Al Abdi | Saad Ezzini
Recent advances in Large Language Models (LLMs) have shown their strong abilities in several tasks, such as text understanding and generation. In several domains, such as software engineering, LLMs are increasingly used to automate related tasks. However, LLM-generated outputs still raise major concerns about robustness and reliability, especially in real-world use. These concerns are especially important in cases where the available inputs are informal natural language rather than structured inputs, which raises issues related to the trustworthiness and potential risks of LLM outputs. This paper aims to evaluate LLMs for the automated test case generation task in a weaker-input setting, relying solely on their understanding of these unstructured inputs. A dataset of 191 programming problems is used to evaluate a general-purpose LLM, GPT-5-mini, and a code-specialized LLM, Qwen2.5-Coder-7B. The generated test cases are executed on reference Python solutions and evaluated in terms of both test-level and problem-level pass rates. The results show that GPT-5-mini outperformed Qwen2.5-Coder-7B in terms of both performance metrics, with 63.72% and 59.16% for test-level and problem-level, respectively, while Qwen2.5-Coder-7B achieves only 21.62% and 2.09% test-level and problem-level pass rates, respectively. The results of this study indicate that both models have limitations in understanding the semantics of informal natural language descriptions, and provide early insight into the feasibility and limitations of LLM-based test case generation using only natural language descriptions. These results also raise concerns related to the safe deployment of LLMs and potential risks, including undetected incorrect interpretations, especially in automated workflows and safety-critical applications.
Fine-Tuning Small Language Models for Cybersecurity: Data Ordering, Knowledge Distillation, and the Educator Effect
Ozkan Kilic | Raja Soundaramourty | Ramu Chenchaiah
Ozkan Kilic | Raja Soundaramourty | Ramu Chenchaiah
Data-sovereignty rules forbid cloud-hosted AI in many high-security environments, leaving compact on-premise models as the only path to AI-assisted cybersecurity. We fine-tune three small open-source models, Gemma 2 2B, Phi-3.5 3.8B, Llama 3.1 8B, on ~147,600 synthetic cybersecurity QA pairs using QLoRA on V100 GPUs. Under strict MCQ evaluation Gemma 2 gains +9.3 pp, Phi +4.0 pp, and Llama drops −24.0 pp. We term this the educator effect: models trained on pedagogical data internalize explanatory behavior at the expense of format compliance. Severity appears to scale with capacity, though capacity is confounded with architecture and learning rate. A controlled ablation shows randomized ordering outperforms curriculum, without significance testing on the 75-question exam.
Large-Scale Multilingual SMS Fraud Detection For Telecom Networks
Orlando Amaral Cejas | Yuejun Guo | Qiang Tang
Orlando Amaral Cejas | Yuejun Guo | Qiang Tang
Short Message Service is a fundamental communication channel in modern telecom networks, yet its ubiquity continues to be exploited for large-scale attacks. Recent advances in multilingual embeddings and large language models have shown strong performance on general text classification tasks. However, their effectiveness and efficiency for multilingual SMS fraud detection under constraints of real telecom environments, remain underexplored. In particular, existing studies largely focus on monolingual datasets, cloud-based inference, or overlook the constraints imposed by real telecom environments. In this work, we investigate large-scale multilingual SMS classification for HAM, SPAM, and SMISHING detection using embedding models. We construct and curate a proprietary multilingual SMS dataset and conduct a systematic evaluation of four different multilingual embedding models. Using the obtained dataset, we fine-tune the models, demonstrating that domain-adapted embeddings significantly improve SMS classification across several languages. Overall, this study addresses the gap between embedding-centric NLP research and real-world telecom requirements, providing empirical and practical insights for effective and efficient deployment of multilingual SMS fraud detection systems. We open source all non-proprietary material at: https://figshare.com/s/1df3ca8d08a4eb4d2712.
Command-Line Obfuscation Detection in Real-World Telemetry under Extreme Class Imbalance
Vojtěch Outrata | Barbora Štěpánková | Michael Adam Polák | Martin Kopp
Vojtěch Outrata | Barbora Štěpánková | Michael Adam Polák | Martin Kopp
To avoid detection by endpoint security tools, adversaries employ command-line obfuscation to alter syntax while preserving functionality. This paper proposes a scalable detection method specifically for command-line data, centered on a custom-trained, small transformer-based model optimized for low-latency inference across massive data streams. We demonstrate the method’s efficacy through a two-phase evaluation: first, by benchmarking the model on a controlled dataset simulating realistic command-line telemetry with extreme class imbalance, where it outperforms previous approaches. Second, we evaluate the model against multiple days of high-volume telemetry from diverse real-world environments. Our results show that this approach provides the high precision and computational efficiency required to handle large-scale command-line logs while effectively reducing analyst workload.
OBSIDIAN: An OSINT-Driven NLP Framework for Detecting Cyber-Physical Threats in Arabic Social Media
Abdullah Saeed Almalki | Salmane Chafik | Saad Ezzini
Abdullah Saeed Almalki | Salmane Chafik | Saad Ezzini
Social networks have evolved into rich sources of Open-Source Intelligence (OSINT), enabling analysts to monitor unrestrained content expressing user activities, sentiments, and emerging behaviors. The immense use of these platforms has made it essential for cybersecurity and threat intelligence professionals to analyze and classify such content to proactively detect cyber-physical threats. While significant research has been conducted on the Arabic language regarding Hate Speech (HS) and Cyberbullying (CB), limited work has addressed Cyber Threat Intelligence (CTI) and OSINT-driven security classification in Arabic, despite their critical importance for early-warning systems and crisis response. In this paper, we introduce OBSIDIAN-AR, a novel real-world, large-scale dataset designed for detecting cyber-physical threats, comprising over 15,000 social media posts primarily from the Gulf region. The dataset is manually curated and annotated into five OSINT-relevant categories: Violence, Threat, Distress, Complaint, and Neutral. Using OBSIDIAN-AR, we fine-tune an Arabic BERT-based model named OBSIDIAN. This context-aware framework acts as a digital early-warning system, leveraging pretrained language representations and domain-specific knowledge derived from our high-quality dataset. Experimental results demonstrate that OBSIDIAN achieves strong performance, reaching up to 98% accuracy on unseen data, proving its viability for modern cybersecurity operations.
Bloc-Conditional Event States: Measuring Cross-Coverage Divergence for Threat-Intelligence Analysis
Maryam Fooladi | Federico Bottino
Maryam Fooladi | Federico Bottino
We propose a content-level measurement of cross-bloc framing divergence in news coverage of contested events, built on the eventstate (ρe) formalism of Bottino et al. (2026). For a given event, outlets are aggregated into editorially-coherent blocs and each bloc is represented by a density matrix ρ bloc on a 15-dimensional framing space. The trace distance D(ρ state, ρmainstream) measures cross-bloc divergence; benchmarking it against the withinWestern polarization D(ρ right, ρleft) controls for editorial variation. The top eigenvector of (ρ state − ρ mainstream) attributes divergence to specific framing axes. Two case studies (Hormuz blockade 2026, n = 16; Navalny death 2024, n = 14) demonstrate the construction. The work positions D(ρ state, ρmainstream) as a content-level observable of potential interest to threat-intelligence workflows that currently rely on source-level features.
CSULoRA: Closest Safe Update Low-Rank Adaptation
Oleksandr Marchenko | Adelaide Danilov | Aria Nourbakhsh | Salima Lamsiyah
Oleksandr Marchenko | Adelaide Danilov | Aria Nourbakhsh | Salima Lamsiyah
Low-rank adaptation has become a standard method for parameter-efficient fine-tuning of large language models, but even small amounts of unsafe or adversarial fine-tuning data can substantially weaken the safety behavior of aligned models. Existing safety-preserving LoRA methods often rely on hard interventions such as projection, pruning, thresholding, or additional training objectives. While these methods can suppress unsafe update directions, they may also remove task-relevant information or require extra tuning. We introduce CSULoRA, a post-hoc method for correcting trained LoRA adapters through closest safe update estimation. CSULoRA estimates a safety-aligned subspace from the weight displacement between a safety-aligned model and its corresponding base checkpoint. It then decomposes each LoRA update into fully aligned, partially aligned, and off-subspace components. Instead of discarding components outside the estimated safety subspace, CSULoRA solves a closed-form penalized minimum-change problem that preserves the fully aligned component while smoothly attenuating potentially unsafe directions according to their relative energy. In adversarial fine-tuning experiments, CSULoRA substantially reduces attack success rate while preserving most of the utility gains obtained from standard LoRA fine-tuning[<https://github.com/Oleksandr-MB/NLPAICS2026_CSULoRA>].
LLM-based Defense Against Adversarial Abstracts in ML/AI Conference Reviewer Assignments
Mohamed Omar Cherif | Christophe Cerisara | Julien Falgas
Mohamed Omar Cherif | Christophe Cerisara | Julien Falgas
Large Machine Learning Conferences guarantee quality reviews of submitted papers by assigning each submission to reviewers who are experts in the relevant topics. This is typically realized by matching reviewers’ expertise to the paper abstract with text semantic embeddings. In order to try and maximize their acceptance score, malevolent authors may attack this assignment process by modifying their abstract so that it matches the expertise of colluding accomplices registered as reviewers. The success of such attacks has been recently demonstrated for realistic conference reviewing datasets with SPECTER embeddings. We propose in this work a defense mechanism against such attacks that leverages Large Language Models (LLM) to rewrite the submitted abstracts and remove the targeted alteration of the original abstract that were aimed at the colluding reviewers. We demonstrate experimentally the effectiveness of our defense that prevents assigning most malevolent abstracts to their colluding reviewer, while preserving the topic-based assignment of normal abstracts to expert reviewers.
From Detection to Attribution: Forensic Linguistics and Adversarial Red Teaming as Complementary Responses to LLM Misuse
Rui Sousa-Silva
Rui Sousa-Silva
The proliferation of Large Language Models (LLMs) has enabled the automation of cyber-attacks (including phishing, social engineering, and impersonation) at unprecedented scale, while existing safeguards remain routinely circumvented. Current detection approaches, predominantly based on stylometric and machine learning methods, face fundamental limitations against adaptive adversaries and struggle with the implicit, contextual, and pragmatic dimensions of language. This article proposes forensic linguistic analysis grounded in the theory of idiolect as a complementary approach to LLM-generated text detection and attribution. We adopt a red teaming methodology to generate synthetic toxic texts that bypass model guardrails to create controlled conditions for testing whether qualitative forensic analysis can succeed where quantitative approaches falter. The findings of our stylometric, character n-gram, and cluster analysis converge to provide moderate evidence that stylometric approaches succeed in discriminating authorship. However, they are not conclusive and hence fall short of current admissibility criteria across diverse jurisdictions. The article thus concludes that idiolect-based forensic analysis can distinguish genuine authorship from LLM-generated impersonation, even when surface features are manipulated. We discuss implications for legal and investigative contexts, where interpretable, theoretically-grounded expert analysis is required over black-box classifier outputs.
The Human Attack Surface: Detecting Psychological Vulnerabilities to Cyber Threats using AI and Social Media
Aadil Gani Ganie | Saad Ezzini
Aadil Gani Ganie | Saad Ezzini
Mental health is a crucial factor influencing overall human well-being, and in the digital age, psychological vulnerabilities have increasingly become a critical human attack surface for cyber threats. Individuals experiencing mental health disorders such as depression or anxiety are demonstrably more susceptible to targeted cyber exploitation, including social engineering, phishing, and digital coercion. In response to this intersection of psychological well-being and human-centric cybersecurity, this study explores the use of social media data for the prediction and classification of mental health conditions to identify vulnerable populations, alongside the development of a secure AI-assisted support system. We analyze and combine three datasets to construct a unified dataset of 14 classes, enabling fine-grained risk assessment for conditions including depression, anxiety, and suicidal ideation. For classification, Logistic Regression significantly outperforms Multinomial Naive Bayes, achieving an accuracy of 95% compared to 74%. To bridge the gap between detection and intervention, the system integrates a conversational module powered by LLaMA 2 (7B). Activated when severe risk is detected, this module provides context-aware interaction to support individuals and mitigate their vulnerability to both psychological crisis and digital exploitation. Deployed via Streamlit, this research serves as an assistive tool for professionals, highlighting the potential of combining machine learning with conversational AI to secure the human element against multifaceted digital and cognitive threats.
A Neuro-Cyber Exploitation and Reconnaissance Taxonomy (NeuroCERT) for Human-Centric Cybersecurity
Cengiz Acarturk | Melike Çağlayan | Ece Caglayan | Anna Wilkosz
Cengiz Acarturk | Melike Çağlayan | Ece Caglayan | Anna Wilkosz
Neuroscientists and cyber threat actors operate on a "black box" paradigm, relying on both the passive measurement of physical exhaust to infer hidden internal states, and the active injection of targeted stimuli to disrupt or manipulate those systems. This paper introduces the Neuro-Cyber Exploitation and Reconnaissance Taxonomy (NeuroCERT), a theoretical framework that maps cybersecurity side-channel attacks and active vectors to psychophysiological measurement and neurostimulation techniques. It proposes that AI can serve as a translational layer to parse noisy biological data into actionable metrics or automate closed-loop attacks, suggesting the potential of novel threat vectors in human-centric cybersecurity.
Analysing Self-Harm Representations in Language Models: a Cross-Architecture Study
Luis Espinosa Anke | Carla Perez Almendros
Luis Espinosa Anke | Carla Perez Almendros
Self-harm content is particularly challenging to detect using NLP techniques, and is also a high-stakes task which requires the highest accuracy to enable timely intervention or flagging at-risk users. We therefore present an analysis of how LLMs represent such self-harm content, which has downstream applications in self-harm detection, LLM intervention and governance and policing. In this paper, we focus on two datasets and four models, and perform two main experiments: (1) We train and evaluate linear probes across all layers of each model on two self-harm datasets: X-Sensitive and SH-Detection. Across both corpora, self-harm information crystallizes in the final 3 - 7% of network layers (93 to 97% depth). (2) We extract contrastive self-harm directions and, after performing a normaliation step, we find that the most accurate probes are not necessarily the most linearly separable. In particular, we find Gemma-3-4B to represent this contrastive self-harm direction in a slightly different, more intricate way than the other LLMs.
Large Language Models (LLMs) are increasingly deployed across a wide range of applications, from conversational assistants to decision support systems. However, these systems remain vulnerable to prompt injection attacks, in which carefully crafted inputs manipulate model behavior and circumvent intended safeguards. While existing research has largely approached prompt injection as a technical or security problem, the linguistic mechanisms through which such attacks operate remain insufficiently understood. In this paper, we argue that prompt injection attacks are fundamentally linguistic in nature, exploiting discourse structure, pragmatic framing, and instruction hierarchies encoded in natural language prompts. Drawing on concepts from speech act theory, discourse analysis, and pragmatics, we propose a typology of four linguistic strategies used to manipulate Large Language Models: instruction override, role framing, hypothetical framing, and procedural prompting. Through detailed linguistic analysis of representative examples, we demonstrate how each strategy exploits identifiable properties of natural language interaction to reshape model interpretation and influence output generation. For each strategy, we also discuss implications for detection and mitigation, arguing that effective safeguards must attend to discourse-level patterns in prompts rather than relying solely on surface-level keyword filtering. Our findings contribute to emerging research at the intersection of computational linguistics and AI security and highlight the importance of integrating linguistic expertise into the design of more robust and reliable language-based AI systems.
How Well Do Commodity Text-to-Speech Systems Evade Acoustic Perturbation Detection? A Multi-Engine Evaluation Across 21 Languages
Anatoly Marchenko
Anatoly Marchenko
Jitter, shimmer, and harmonics-to-noise ratio (HNR) are often used to detect voice deepfakes, since these features capture biomechanical irregularities of vocal fold vibration that synthetic speech supposedly lacks. We test this assumption on three commodity TTS engines (Google TTS, Microsoft Edge TTS, macOS system voice) with 1,850 samples across 21 languages, measured against 29 emotion corpora in 24 languages (35,091 utterances). Three classifiers (logistic regression, SVM-RBF, Random Forest) all fail to reliably detect Edge TTS: the best result is F1 = 0.78. Effect sizes drop 2.1x-7.4x from Google TTS to Edge TTS. In ablation, no single feature exceeds F1 = 0.60 against Edge TTS. These three perturbation features, taken alone, can no longer separate commodity neural TTS from natural speech.
Semantic Clustering of Obfuscated Command-Line Detections for Alert Reduction
Barbora Štěpánková | Vojtěch Outrata | Martin Kopp
Barbora Štěpánková | Vojtěch Outrata | Martin Kopp
We propose a post-processing method for grouping large volumes of command-line detections into semantically coherent cluster-level alerts. The approach combines embedding-based clustering with LLM-based cluster-level filtering: command-lines are first encoded using Sentence-BERT embeddings and grouped via hierarchical agglomerative clustering, after which a large language model evaluates cluster representatives to reduce the number of false positive alerts. We evaluate the method on real-world telemetry from a commercial endpoint protection system, applying it to detections produced by an obfuscation detection model. On one week of data, the pipeline reduces alert volume by approximately 98% while maintaining high cluster purity and semantic coherence, demonstrating its effectiveness as a scalable post-processing step in high-volume detection settings.
Judging the LLM Judges: A Human-Centric Validation of LLM-Generated Training Data for Software Retrieval
Ogtay Hasanov | Saad Ezzini
Ogtay Hasanov | Saad Ezzini
As Large Language Models (LLMs) increasingly generate training data for downstream machine learning systems, the quality of this synthetic data becomes a critical security concern. Low-quality synthetic training data can silently poison retrieval systems deployed in security-sensitive contexts such as software issue triage, user support, and threat intelligence matching. We present a multi-dimensional quality assessment protocol for LLM-generated synthetic training data and apply it to a case study involving 13,579 synthetic user reviews generated from GitHub issues across four open-source Android applications. We evaluate 400 stratified samples using an LLM judge (GPT-4o-mini) along a five-point rubric, find that 10.5% of generated reviews fail to meaningfully capture their source issues, and identify systematic failure patterns concentrated in developer-internal issues (continuous integration, refactoring) and sarcastic persona framings. To validate the LLM judge against human annotation, we compute Cohen’s Kappa between one human rater, a second independent human rater, and the LLM judge on 20 stratified reviews. Our results highlight the need for hybrid human-AI protocols when assessing synthetic data quality for security-critical applications.
Exploring the Manifestation of Schwartz’s Basic Human Values in Large Language Models
Ryan Hyland | Lewis Newsham | Daniel Prince
Ryan Hyland | Lewis Newsham | Daniel Prince
This study investigates the manifestation of Schwartz’s Theory of Basic Human Values (STBV) within a pre-trained Large Language Model (LLM) by evaluating how persona prompting influences the model’s responses. Specifically, we use prompt-based persona induction to represent Schwartz’s ten broad value types e.g., Universalism, Achievement) and measure their effects using the model’s responses to the Portrait Values Questionnaire Revised (PVQ-RR). A neutral persona and human baseline serve as control conditions to assess the influence of persona prompting. Prompting strategies are also compared. Results show that value-based persona prompts systematically shift the model’s PVQ-RR response profiles, indicating that LLM questionnaire responses can be steered along Schwartz value dimensions under controlled prompting conditions. These findings suggest that value-based persona prompts may be useful for studying and configuring the expressed response profiles of LLM-based agents. Using Schwartz’s values as a structured measurement framework provides a way to evaluate how LLM responses change under persona prompting and offers a basis for future studies of value-conditioned agent behaviour.
From Decision Tree to Detection Pipeline: Formalizing van Dijk’s Socio-Cognitive Framework for Automated Anti-Language Identification in RICO Transcripts
Elena Morandini
Elena Morandini
This paper proposes a context-first NLP detection pipeline for automated anti-language identification in RICO wiretap transcripts. Current threat-detection classifiers fail on organized crime discourse because criminal intent is encoded through implicature and relexicalization rather than explicit lexical markers. The pipeline addresses this architectural mismatch by formalizing van Dijk’s (2011) socio-cognitive CDA framework as a sequential seven-step decision tree mapped to concrete NLP subtasks: from speaker-role classification and genre detection to deontic feature extraction and ensemble scoring. A six-feature micro-level vector (F1–F6), validated against a 14,072-word corpus of authenticated Mafia communications, operationalizes the ideological square as a two-axis feature space that measures discursive distance between the ingroup and the outgroup. Preliminary evaluation confirms statistically significant patterns (χ² = 90.82, p < 0.001 for pragmatic divergence; 4:1 deontic saturation ratio) consistent with anti-language characteristics. The pipeline enables three LEA applications: automated flagging, context-sensitive decoding, and communication network analysis. Ethical considerations regarding false positives, privacy, and evidentiary standards are discussed.
When Privacy Helps: Pseudonymisation as a Strategy for Improved Cyber Incident Classification
Loya C. Haughton | Eduardo Fidalgo | Rocío Alaiz-Rodríguez | Manuel Castejón-Limas | Laura Fernández-Robles
Loya C. Haughton | Eduardo Fidalgo | Rocío Alaiz-Rodríguez | Manuel Castejón-Limas | Laura Fernández-Robles
Organisations often rely on Cyber Threat Intelligence (CTI) for collective defence against evolving threats. Therefore, it is important that these shared reports are both compliant with data protection legislation and analytically useful. Pseudonymisation is a recognised data protection measure, yet its effect on downstream utility remains underexplored in cybersecurity. Unlike other domains where personal identifiers carry predictive value, personal data in cyber incident reports may act as noise rather than signal — suggesting pseudonymisation could improve classification accuracy. This hypothesis is evaluated in two steps. First, we derive three new datasets by applying Data Masking, Data Tokenisation and Data Substitution to a subset of the CECILIA-10C-900 dataset. We then evaluate 21 models — spanning traditional Machine Learning (ML) classifiers, encoder transformers and QLoRA fine-tuned Large Language Models (LLMs) — on these datasets, for a CTI classification task based on the Spanish National Cybersecurity Institute’s incident taxonomy. The RoBERTa-base model achieved the highest overall weighted F1-score of 87.35% when Data Tokenisation was applied, while Llama-3.1-8B demonstrated the largest gain (+12.63 pp) with Data Masking. These findings reframe pseudonymisation from only a compliance measure into a preprocessing step that may simultaneously protect privacy and improve classification in specific model-technique pairings.
Does Hate Transfer? Cross-Lingual Generalisation of Offensive Content Detection Across Indic Languages
Purandhar M. Reddy | Sara Renjit
Purandhar M. Reddy | Sara Renjit
A common assumption in low-resource NLP is that cross-lingual transfer from a related language can substitute for target-language annotation when labelled data is scarce. We test this assumption for offensive content detection across five Indic languages by evaluating all twenty directed transfer pairs from a LLaMA3.1-8B model fine-tuned with Low-Rank Adaptation (LoRA) on the MACD benchmark. Only three of twenty pairs achieve tolerable transfer loss below 15%, all involving Malayalam as the source language. Telugu is the hardest transfer target (average loss 33.8%), while Malayalam is the most transferable source (average loss 16.8%). Confusion-matrix analysis reveals two distinct failure modes: Tamiland Kannada-trained models are conservative under-flaggers that miss 73–82% of offensive content with near-zero false alarms, while Malayalam-trained models are aggressive flaggers that miss far less (39%) but over-flag at 21%. These patterns do not follow typological structure: a Spearman correlation between URIEL typological similarity and transfer F1 yields ρ = −0.254 (p = 0.281), failing to conf irm the typological hypothesis. Our results indicate that cross-lingual shortcuts are unreliable for this task and that language-specific annotation cannot be avoided by appealing to linguistic family membership.
LLMs in the Enterprise: A Systematic Review of Security Architectures for RAG-Augmented Chatbots
Fatimah A. Alali | Haya Aldawsari | Saad Ezzini | Sajjad Mahmood
Fatimah A. Alali | Haya Aldawsari | Saad Ezzini | Sajjad Mahmood
Enterprise sectors increasingly require AI-driven solutions that handle large volumes of domain-specific documentation securely and efficiently. While Retrieval-Augmented Generation (RAG) extends the capabilities of Large Language Models (LLMs) by grounding responses in external knowledge bases, the security and adoption concerns that define industrial deployment remain largely unaddressed. A Systematic Literature Review (SLR) was conducted targeting empirical studies published between 2020 and 2025 across five academic databases, guided by research questions covering RAG performance, industrial applicability, and security requirements. While findings indicate RAG outperforms base LLM configurations, the analysis reveals that security coverage across the corpus is limited and inconsistent. A critical, recurring gap identified is that security architecture is not adequately covered by the current work, with studies failing to provide detailed security architecture guidance. The literature has not yet addressed the complete set of enterprise deployment requirements, particularly those relating to data privacy. We conclude that while RAG provides a strong foundation, establishing robust security architectures and privacy guidelines are the primary directions required for future research and safe enterprise adoption.
Partner Attribution Bias in LLM-Assisted Export Control Screening
Salem Alotaibi | Alexei Lisitsa | Antony McCabe
Salem Alotaibi | Alexei Lisitsa | Antony McCabe
Abstract Automating export control screening with large language models (LLMs) introduces risks of contextual sensitivity to non-technical signals, including partner country identity. This paper presents a reproducible counterfactual framework for diagnosing how partner country information influences LLM compliance decisions across five instruction-tuned models and 249 jurisdictions. Across four controlled experiments, partner identity systematically alters export control classifications even when project content is held constant or removed entirely, with BRICS and GCC partners attracting consistently higher control rates than EU and Five Eyes collaborations. Under causal partner controls, classification deltas become small and statistically non-significant, suggesting that observed disparities reflect contextual cue sensitivity rather than fixed geopolitical bias, though residual associative priors cannot be fully excluded. Model-level analysis reveals qualitatively distinct behavioural patterns, including directionally opposite responses to identical inputs across architectures, indicating that model selection carries consequences beyond accuracy in compliance-sensitive deployments. A structured two-stage prompting protocol reduces decision volatility by 60–80% while preserving interpretability. The proposed framework provides an auditable diagnostic method for evaluating contextual sensitivity in regulated AI systems and offers practical guidance for accountable compliance automation.
Privacy VITA: a new multilingual and multimodal annotated video corpus to evaluate anonymization systems
Jorge Rico | Sofia Contreras | Enrique Manjavacas Arevalo | Maria Viana | Ruben Pérez-Ramón | Maria Luisa Izquierdo | María José Vilella | Jaime Corton | Silvia Rodriguez | Fernando Espinza | Rafael Ginard | César Pérez | Jesús Arias | Richard Cook | Pablo Regodón | Manuel Moyano | Ricardo Heredia | Lara De Santos | Pierre Plaza | Jacqueline González
Jorge Rico | Sofia Contreras | Enrique Manjavacas Arevalo | Maria Viana | Ruben Pérez-Ramón | Maria Luisa Izquierdo | María José Vilella | Jaime Corton | Silvia Rodriguez | Fernando Espinza | Rafael Ginard | César Pérez | Jesús Arias | Richard Cook | Pablo Regodón | Manuel Moyano | Ricardo Heredia | Lara De Santos | Pierre Plaza | Jacqueline González
In this paper, we introduce a new multimodal and multilingual public resource for evaluating anonymization models and solutions, comprising a collection of 516 annotated videos. Unlike previously available resources, the Privacy VITA corpus is both multilingual and multimodal, covering Video, Image, Text and Audio modalities. The development of this dataset is motivated by the growing need to anonymize private data in an increasingly multimedia-driven society. Furthermore, emerging regulations pose significant challenges for organizations and companies that manage sensitive information while ensuring legal compliance. This paper details the processes of collection, preprocessing, annotation, and curation of the corpus, as well as its overall scope. We hope this resource will contribute to the advancement of multimodal anonymization research.