Frontier large language models (LLMs) safety evaluation has largely treated harmful generation as an attack outcome rather than an object of analysis. Consequently, little is known about the outputs themselves, partly because large-scale, high-quality collections of frontier-model misbehavior are difficult to obtain. To address this gap, we introduce **HarmProfile**, a content-centric benchmark dataset that collects model misbehavior across diverse harm categories and model families, and defines the resulting content distribution as a model-level risk profile. The premise is that, just as linguistic behavior can be characterized from an utterance corpus, model risk can be characterized from the content, severity, and variation of its safety failures. HarmProfile contains over 80,000 validated artifacts from 23 frontier LLMs across 13 model families, organized into 15 harm categories and 57 subcategories. Using this corpus, we find that frontier LLMs reliably produces harmful content at scale, yet exhibit distinct risk profiles; both harmfulness and diversity grow with model capability, suggesting that frontier LLMs may appear safe yet harbor increasingly dangerous knowledge beneath the alignment surface. The code and dataset resources are publicly available at https://github.com/fresh-ma/HarmProfile.

Backdoor attacks are an important safety risk for large language models (LLMs), yet existing studies often rely on manually crafted triggers and fixed poisoning pipelines. This limits the diversity of attack behaviors and makes it difficult to systematically evaluate defense robustness. We introduce AutoBackdoor, an agent-driven framework for scalable LLM backdoor red-teaming. Given a target scenario, AutoBackdoor automatically generates triggers, constructs poisoned samples, refines them through self-feedback, and produces backdoored models through fine-tuning. We evaluate the framework across four realistic scenarios: Bias Recommendation, Hallucination Injection, Error Code Generation, and Peer Review Manipulation. Experiments on LLaMA-3, Mistral, Qwen, and GPT-4o show that AutoBackdoor can achieve over 90% attack success with only a small number of poisoned samples. We further find that existing defenses provide limited protection against these agent-generated attacks. These results suggest that automated red-teaming can serve as a useful tool for stress-testing LLM backdoor defenses and understanding emerging risks from agent-driven model customization pipelines. Our code is available at https://github.com/bboylyg/AutoBackdoor.

Object hallucination remains a persistent reliability issue in large vision-language models, where generated object mentions may sound plausible but lack visual grounding. Recent training-free detectors use internal signals such as token likelihood, attention, visual confidence, or image-text similarity to identify hallucinated objects. These signals are useful, but they are often source-confounded. They measure how strongly an object is supported inside the model without distinguishing whether that support comes from object-specific visual evidence or the generated text prefix. In difficult cases, a hallucinated object can still receive high internal support because it fits the scene, is associated with nearby visual cues, or follows naturally from the generated text prefix. We propose VisER, a training-free two-sided metric for object-level hallucination detection. VisER evaluates each generated object mention from two complementary views. Visual Evidence measures whether object-context compatibility is backed by object-specific evidence from image tokens. Visual Reliance measures whether the object is supported more by the image than by the generated prefix. Combining these views gives a more source-aware grounding score, while avoiding additional object-level verification generations. Across multiple LVLMs and benchmarks, VisER improves AUROC and AUPR over a range of baselines.