Pretraining corpus composition shapes LLM capabilities, but it often remains hidden even when model weights are released. Prior work has inferred corpus mixtures or traced specific token groups from released tokenizer vocabularies; in contrast, we estimate corpus ratios for arbitrary target tokens. We first show that BPE tokenizers trained on different corpora share stable token ID–ratio distributions, motivating distribution transfer from known corpora to a target tokenizer trained on hidden corpora. We then propose Quantile-Guided Density Estimation (QGDE), which approximates this distribution with multiple quantile trends and uses local density weighting to produce token-level estimates. In controlled settings and a realistic setting using the released SmolLM tokenizer, QGDE achieves mean relative errors as low as 3.00% for token-level estimation and 3.08% after aggregation into category-level mixtures. These results suggest that released tokenizer vocabularies provide a useful signal for fine-grained corpus estimation beyond coarse composition inference.

Multilingual safety guardrails are essential for deployed LLMs, yet they perform poorly on low-resource languages. Prior work typically relies on large-scale multilingual safety data synthesis, which incurs substantial training costs. Cross-lingual transfer offers a cheaper alternative using limited parallel data, and recent work shows that aligning only last-token hidden states can improve multilingual safety. However, although effective for query-only (Q) risk detection, last-token alignment consistently degrades query-response (QR) detection in multilingual settings, limiting its applicability in practical guardrail systems where Q and QR serve as pre-generation and post-generation safety barriers, respectively. In this paper, we propose Q-Align, a method that decouples query intent from response evaluation by aligning only at the query boundary. Specifically, for each parallel sample, Q-Align extracts the hidden state right after the query and aligns multilingual query representations toward a confidence-weighted anchor, allowing languages with stronger intent understanding to guide transfer. We further apply language grouping based on geographic proximity to stabilize cross-lingual alignment. Experiments demonstrate that Q-Align consistently improves multilingual safety detection in both Q and QR scenarios, raising QR average F1 from 74.70% to 80.79% on Polyguard-Qwen, while preserving English performance and generalizing to languages unseen during training.

Chinese web pollution has surfaced in LLMs, motivating audits of upstream Chinese corpora. However, auditing such corpora faces three challenges: (1) their web-scale size makes full scan costly; (2) prior analyses are often too coarse to expose token-level pollution; (3) Chinese web pollution is implicit and rapidly changing. We propose Sampled-BPE, a lightweight token-level auditing pipeline that sample a small subset and train BPE tokenizer to surface polluted tokens. Experiments show that Sampled-BPE preserves usable estimates while substantially reducing runtime and memory: a 148.4 × speedup and a 35.8 × memory reduction induce only 4.25% relative error for pollution categories. We apply the pipeline to 11 open Chinese corpora and 6 Chinese Common Crawl snapshots from 2021 to 2026. The audit reveals widespread but uneven pollution across open corpora, as well as highly polluted and temporally shifting Chinese web content. We further release a hierarchical Chinese web token dataset with 660k+ token records, each with web context, category, and explanation fields, organized as trees to support review and tracing of pollution.

Jailbreaks against multimodal large language models (MLLMs) typically operate through image-text inputs. After each target response, the attacker must decide how the new feedback should shape the next image-text query: which clues should be stated in the text, which should appear as visual evidence, and how the text should refer to the image.Existing multimodal attacks typically rely on fixed image-text pairs or candidate search, limiting their ability to adapt across turns.Text-only multi-turn attacks can adapt to prior responses, but they do not decide how response-derived clues should be expressed across image and text.In this paper, we formulate this missing step as trace-to-query realization and propose  ̲Memory- ̲Augmented  ̲Visual  ̲Querying (MAViQ), a black-box framework that converts response-derived clues into visual and textual constraints, refines the query after observing the generated image, and propagates feedback to subsequent turns.Extensive experiments on two benchmarks across six target MLLMs show that MAViQ achieves average attack success rates (ASRs) of 70.60% on MM-SafetyBench and 82.13% on JailbreakV-mini, outperforming the strongest baseline by 5.90% and 8.32%, respectively. Our code is available at https://github.com/jixingtian/Maviq.

Due to the absence of connectives, implicit discourse relation recognition (IDRR) is still a challenging and crucial task in discourse analysis. Most of the current work adopted multitask learning to aid IDRR through explicit discourse relation recognition (EDRR) or utilized dependencies between discourse relation labels to constrain model predictions. But these methods still performed poorly on fine-grained IDRR and even utterly misidentified on most of the few-shot discourse relation classes. To address these problems, we propose a novel Prompt-based Connective Prediction (PCP) method for IDRR. Our method instructs large-scale pre-trained models to use knowledge relevant to discourse relation and utilizes the strong correlation between connectives and discourse relation to help the model recognize implicit discourse relations. Experimental results show that our method surpasses the current state-of-the-art model and achieves significant improvements on those fine-grained few-shot discourse relation. Moreover, our approach is able to be transferred to EDRR and obtain acceptable results. Our code is released in https://github.com/zh-i9/PCP-for-IDRR.