We study the vulnerability of guardrail models to adversarial paraphrases that exploit superficial textual variation: meaning-preserving changes that alter expressions spuriously correlated with safety labels. While prior work probes this vulnerability through simple heuristics, such as keyword-label co-occurrence statistics, we argue that guardrails may rely on more complex textual expressions involving phrases or syntactic structures that cannot be captured by heuristics. To enable systematic evaluation of such hidden vulnerabilities, we propose ASPro, an agentic adversarial evaluation framework that automatically synthesizes meaning-preserving paraphrases containing expressions that contradict guardrails’ learned spurious expression-label correlations. Applying ASPro to five safety benchmarks and 13 guardrail models, we find that the average F1 drops from 77.83 to 40.25, substantially more than under existing heuristic baselines. Moreover, on ASPro prompts, guardrail models often fail to outperform their base LLMs before safety fine-tuning. These results suggest that current safety fine-tuning may overfit to spurious expressions rather than truly improve guardrails’ understanding of prompt safety.

Translating sequential programming priors into the parallel temporal logic of hardware design remains a crucial bottleneck for large language models (LLMs). To investigate this, we introduce an operational error taxonomy grounded in the Register-Transfer Level (RTL) coding pipeline. Our taxonomy categorizes failures into extraction/syntax (L1), compilation/elaboration (L2), and testbench-observed functional (L3) stages under a fixed evaluator. Evaluations reveal an empirical ceiling on the VerilogEval benchmark, as frontier models plateau at a 90.8% initial pass rate. Furthermore, we expose a striking surface convergence gap: optimization readily eliminates syntax errors but concurrently exacerbates deeper functional failures. Our findings demonstrate that alignment techniques merely teach models to compile. While test-time scaling can patch solvable errors, RTL coding capacity remains bounded by pretraining knowledge. Addressing challenges in the current LLM-based hardware generation pipeline requires more studies in model reasoning rather than alignment interventions.

We introduce a voice-agentic framework that learns one critical omni-understanding skill: knowing when to trust itself versus when to consult external audio perception. Our work is motivated by a crucial yet counterintuitive finding: naively fine-tuning an omni-model on both speech recognition and external sound understanding tasks often degrades performance, as the model can be easily misled by noisy hypotheses. To address this, our framework, Speech-Hands, recasts the problem as an explicit self-reflection decision. This learnable reflection primitive proves effective in preventing the model from being derailed by flawed external candidates. We show that this agentic action mechanism generalizes naturally from speech recognition to complex, multiple-choice audio reasoning. Across the OpenASR leaderboard, which includes seven domain-diverse speech datasets, Speech-Hands consistently outperforms strong baselines by 12.1% WER on the OpenASR benchmark. The model also achieves 77.37% accuracy and high F1 on audio QA decisions, showing robust generalization and reliability across diverse audio question answering datasets. By unifying perception and decision-making, our work offers a practical path toward more reliable and resilient audio intelligence.

It is challenging to update Large language models (LLMs) since real-world knowledge evolves. While existing Lifelong Knowledge Editing (LKE) methods efficiently update sequentially incoming edits, they often struggle to precisely overwrite the outdated knowledge with the latest one, resulting in conflicts that hinder LLMs from determining the correct answer. To address this Serial Lifelong Knowledge Editing (sLKE) problem, wepropose a novel Mixture-of-Knowledge-Experts scheme with an Activation-guided Routing Mechanism (ARM), which assigns specialized experts to store domain-specific knowledge and ensures that each update completely overwrites old information with the latest data. Furthermore, we introduce a novel sLKE benchmark where answers to the same concept are updated repeatedly, to assess the ability of editing methods to refresh knowledge accurately. Experimental results on both LKE and sLKE benchmarks show that our ARM performs favorably against SOTA knowledge editing methods.

Large Vision-Language Models (LVLMs) have transformed image captioning, shifting from concise captions to detailed descriptions. We introduce LOTUS, a leaderboard for evaluating detailed captions, addressing three main gaps in existing evaluations: lack of standardized criteria, bias-aware assessments, and user preference considerations. LOTUS comprehensively evaluates various aspects, including caption quality (e.g., alignment, descriptiveness), risks (e.g., hallucination), and societal biases (e.g., gender bias) while enabling preference-oriented evaluations by tailoring criteria to diverse user preferences. Our analysis of recent LVLMs reveals no single model excels across all criteria, while correlations emerge between caption detail and bias risks. Preference-oriented evaluations demonstrate that optimal model selection depends on user priorities.

Construction of a general-purpose post-recognition error corrector poses a crucial question: how can we most effectively train a model on a large mixture of domain datasets? The answer would lie in learning dataset-specific features and digesting their knowledge in a single model. Previous methods achieve this by having separate correction language models, resulting in a significant increase in parameters. In this work, we present Mixture-of-Experts as a solution, highlighting that MoEs are much more than a scalability tool. We propose a Multi-Task Correction MoE, where we train the experts to become an “expert” of speech-to-text, language-to-text and vision-to-text datasets by learning to route each dataset’s tokens to its mapped expert. Experiments on the Open ASR Leaderboard show that we explore a new state-of-the-art performance by achieving an average relative 5.0% WER reduction and substantial improvements in BLEU scores for speech and translation tasks. On zero-shot evaluation, NeKo outperforms GPT-3.5 and Claude-3.5-Sonnet with 15.5% to 27.6% relative WER reduction in the Hyporadise benchmark. NeKo performs competitively on grammar and post-OCR correction as a multi-task model.