Multilingual red-teaming benchmarks often reduce cultural localization to direct translation, overlooking sociotechnical factors shaping AI safety across contexts. By treating culturally distinct contexts as equivalent, this translation-centered approach can flatten regional diversity, obscure diverse perspectives, and reinforce single-ground-truth assumptions in AI safety evaluation. We present LatAm-RT, a context-aware red-teaming benchmark for incorporating Latin American cultural pluralism into AI safety evaluation. Its Layered Localization Taxonomy (LLT) models localization as an explicit experimental variable across three localization levels direct translation (L1), regional adaptation (L2), and national localization (L3) for Mexico, El Salvador, Costa Rica, and Chile while preserving comparable adversarial intent. We evaluate five language models using 284 prompts across three localization layers, four risk domains, and five adversarial framing strategies. Results show that localization is neither uniformly beneficial nor uniformly harmful: country-specific institutional vocabulary is associated with stronger safeguard activation in Privacy and Civic Systems, whereas politically situated narratives retain their adversarial effectiveness in Political and Information Harm. Adversarial framing remains the primary driver of attack success rate (ASR) variation, including a 55-percentage-point gap between GPT-4o and Claude Sonnet 4.6 under defensive framing. LatAm-RT introduces the Localization Effect (LE) to quantify regional robustness differences obscured by translation-based evaluations. These findings motivate pluralistic AI safety evaluation across culturally situated contexts. The dataset and annotation files are available at https://huggingface.co/datasets/Latam26/LatAmRT

Phishing can turn stolen personal information into financial harm, yet some sites hide data collection beyond the landing page. We present a site-level workflow for Korean-language phishing that tracks linked pages and supplies LLMs with relation-centered DOM summaries. The 256 landing URLs treated as phishing in the full collection were labeled primarily by the initial LLM, with human labels taking precedence for reviewed sites. Of these, 161 (62.9%) were detected by at least one VirusTotal engine; 215 of 217 linked URLs (99.1%) had no prior query record, and neither remaining URL was detected. On 70 human-validated landing pages, summarization increased Gemini 2.5 Flash-Lite’s F1 from 0.621 to 0.925, while F1 changes for the other three Gemini models ranged from -0.011 to +0.035. In a six-site case study, data collection was detected only on linked pages, while page-level aggregation retained landing-page impersonation targets more consistently than concatenation. These results highlight the importance of exploration scope and show model-dependent accuracy effects of relation-centered DOM summaries in phishing detection.

Financial institutions must prevent personally identifiable information (PII) and adversarial instructions from entering large language models (LLMs). We present regression-aware policy maintenance for Korean financial LLM input guardrails, supported by a cost-ordered Regex–NER–LLM cascade. On a privacy-preserving controlled set of 1,000 synthetic attacks and 200 routine requests, four manual iterations reduce in-distribution attack success rate (ASR) from 8.9% to 0.3% with a 0.5% false-positive rate (FPR) on the original tuning set. On a held-out split, a policy tuned only on the tuning split achieves 0/300 attack successes and 1/60 false positives, with Wilson 95% CIs of [0, 1.26]% and [0.29, 8.86]%, respectively. For policy maintenance, an LLM generates candidate patches, while paired McNemar tests, an FPR constraint, and cumulative regression checks determine acceptance or rejection. Paired regression testing detects an intermediate regression introduced in manual Round 3, while the gate rejects all five automatically generated candidates, preventing statistically unsupported changes from entering the policy. For boundary diagnosis, the rejected fifth-round candidate yields a pre-mitigation overall FPR of 28.9% across 350 benign examples, including 150 hard negatives, revealing context-blind Regex blocking and LLM-judge over-blocking hidden by routine benign requests. These results support regression-aware policy maintenance in the controlled setting while highlighting the need for broader boundary and deployment-specific evaluation.

Guard models are used as safety classifiers, with confidence scores driving downstream moderation decisions. We evaluate five guard models for prompt classification and find that although several are nearly calibrated on clean inputs, adversarial attacks degrade their calibration by an order of magnitude, turning false negatives into high-confidence errors indistinguishable from correct detections. Comparing each guard with its corresponding base LM, we find that uncertainty signals often remain available, with the base model typically expressing uncertainty on the same inputs where the guard fails. Layer-wise analyses localize this guard–base divergence to later layers, where guard models exhibit sharper safe/unsafe separation and lower-rank representations, while adversarial harmful inputs lie closer to the clean-safe region. These findings highlight a mismatch between guard confidence and base-model uncertainty under attack.

While Large Language Models (LLMs) are widely used, they remain susceptible to jailbreak prompts that can elicit harmful or inappropriate responses. This paper introduces STAR-Teaming, a novel black-box framework for automated red teaming that effectively generates such prompts. STAR-Teaming integrates a Multi-Agent System (MAS) with a Strategy-Response Multiplex Network and employs network-driven optimization to sample effective attack strategies. This network-based approach recasts the intractable high-dimensional embedding space into a tractable structure, yielding two key advantages: it enhances the interpretability of the LLM’s strategic vulnerabilities, and it streamlines the search for effective strategies by organizing the search space into semantic communities, thereby preventing redundant exploration. Empirical results demonstrate that STAR-Teaming significantly surpasses existing methods, achieving a higher attack success rate (ASR) at a lower computational cost. Extensive experiments validate the effectiveness and explainability of the Multiplex Network. The code is available at https://github.com/selectstar-ai/STAR-Teaming-paper.

To reliably deploy Large Language Models (LLMs) in a specific country, they must possess an understanding of the nation’s culture and basic knowledge. To this end, we introduce National Alignment, which measures the alignment between an LLM and a targeted country from two aspects: social value alignment and common knowledge alignment. We constructed KorNAT, the first benchmark that measures national alignment between LLMs and South Korea. KorNat contains 4K and 6K multiple-choice questions for social value and common knowledge, respectively. To attain an appropriately aligned ground truth in the social value dataset, we conducted a large-scale public survey with 6,174 South Koreans. For common knowledge, we created the data based on the South Korea text books and GED exams. Our dataset creation process is meticulously designed based on statistical sampling theory, and we also introduce metrics to measure national alignment, including three variations of social value alignment. We tested seven LLMs and found that only few models passed our reference score, indicating there exists room for improvement. Our dataset has received government approval following an assessment by a government-affiliated organization dedicated to evaluating dataset quality.