Voice phishing is an evolving form of social engineering crime and requires the continuous advancement of detection technologies. We introduce a benchmark dataset designed to evaluate the practical performance of AI-based voice phishing detection models. The dataset includes diverse voice conversation scenarios and supports four evaluation tasks to assess open-source language models. Experimental results show that while some large-scale models demonstrate stable performance across multiple tasks, accuracy remains low in topic classification and dialogue structure recognition, regardless of model size. These findings highlight the complexity of voice phishing detection, which demands contextual reasoning and dialogue structure understanding beyond simple sentence-level comprehension. The proposed benchmark dataset provides a foundation for more robust evaluation and development of AI systems capable of detecting deceptive voice interactions, contributing to safer and more trustworthy communication environments

We propose CAPE, a Choice-Distribution Alignment and Performance Evaluation framework for diagnosing human–LLM response alignment using population-scale human choice distributions. CAPE separates outcome-level difficulty alignment from response-distribution alignment with four metrics: OA, MAE, PA, and DA. We apply CAPE to 609 Korean CSAT items with item-level response distributions from approximately 300K examinees per item, and evaluate 16 LLMs. Results show that model capability is the strongest predictor of alignment, with accuracy closely tied to both full-choice and distractor-only alignment. However, group-level scores are affected by aggregation size and model composition, and neither reasoning augmentation nor Korean adaptation yields consistent residual advantages. Bloom-level analyses further suggest that capability-linked alignment varies across item types. CAPE thus enables evaluation beyond accuracy by diagnosing how model success and failure patterns relate to human population responses.

Current text anonymization evaluation relies on span-based metrics that fail to capture what an adversary could actually infer, and assumes a single data subject, ignoring multi-subject scenarios. To address these limitations, we present SPIA (Subject-level PII Inference Assessment), the first benchmark that shifts the unit of evaluation from text spans to individuals, comprising 675 documents across legal and online domains with novel subject-level protection metrics. Extensive experiments show that even when over 90% of PII spans are masked, subject-level inference protection drops as low as 33%, leaving the majority of personal information recoverable through contextual inference. Furthermore, target-subject-focused anonymization leaves non-target subjects substantially more exposed than the target subject. We show that subject-level inference-based evaluation is essential for ensuring safe text anonymization in real-world settings.