Next-token prediction is the self-supervised signal that trains language models, and every observed prompt token provides the same signal at test time. We study whether this signal can define the inner-loop objective for test-time training (TTT) in pretrained long-context language models. Existing in-place TTT methods store prompt information in fast weights, but they train each write to match a learned shortcut target: an internal pattern that may be easy to predict, yet is not the trajectory the model uses to form its next-token distribution. We introduce Test-Time Training with Next-Token Prediction (TTT-NTP), a drop-in fast-weight adaptation method for pretrained LLMs that instead supervises updates using the model’s own next contextual hidden state. This makes each local write follow the same causal computation that supports next-token prediction, without aggregating over a neighborhood of positions: the value target is a pointwise linear projection of a single next-position contextual state. Across four backbones from three model families, Llama-3.1-8B, Mistral-7B-v0.3, Qwen3-4B, and Qwen3-0.6B, TTT-NTP is the only method that consistently improves RULER Full-13 over the released backbone, by 2.9 to 4.1 points, while aggregate commonsense and knowledge performance is preserved. Our code is publicly available at https://github.com/yancyou/TTT-NTP.

Large language models excel at general tasks but underperform smaller supervised models in specialized, high-stakes domains where training labels are costly. We address this regime with EvoPool, an evolutionary multi-agent framework inspired by Darwinian evolution. Three specialized agents iteratively propose executable annotator code, a small validation set provides a fitness signal, and a deterministic gate keeps only annotators that pass viability, diversity, and marginal-contribution checks across generations. Pool votes are mapped to soft training labels by EvoAgg, a text-aware aggregator combining semantic features with annotator-vote features. The authored pool runs at near-zero per-example cost and is 4500 to 31000× faster than LLM annotation on 100K examples. Across 7 out of 8 LLM-weak specialized and complex tasks spanning biomedical relation extraction, legal-clause classification, complex reasoning, and dense multi-label biomedical classification, EvoPool beats the strongest LLM annotation baseline by an average +0.132 macro-F1, peaking at +0.275 on ChemProt and +0.268 on PubMed. Code is available at: https://github.com/tianyi0216/EvoPool.

Text-to-SQL systems allow users to interact with databases using natural language, but most assume that each question must be answered by a single SQL query. In practical settings, some information needs are more reliably satisfied by generating multiple simpler SQL queries, which is not supported by existing benchmark or evaluation protocols. To address this gap, we introduce Spider𝕊, which, to the best of our knowledge, is the first benchmark for the Text-to-(Multi)-SQL task, supporting both single- and multi-query solutions within a unified evaluation framework. To enable this setting, we propose a flexible evaluation protocol that accepts any prediction covering the required information, regardless of query number or structure. Under our evaluation protocol, we evaluate several large language models on Spider𝕊 and find that allowing multi-query outputs improves execution accuracy. However, current models demonstrate fragility: even a minor prompt change from generating one SQL to generating one or more SQL can significantly degrade performance. To improve robustness, we introduce a lightweight router that decides whether to generate a single or multiple SQL for a given question, leading to substantial performance gains.

Dataset Pruning (DP) aims to construct a coreset that achieves performance comparable to the original, full dataset. However, few studies have explored DP in the context of Speech Classification (SC) tasks. Unlike image or text classification, SC is particularly challenging due to the difficulty in capturing the acoustic, semantic, and contextual representations. In this study, we propose a novel dataset pruning method for speech datasets, termed Meltrim, which uses a two-step coarse-to-fine framework designed to address these challenges. Specifically, in Step 1, Meltrim coarsely filters utterance-level redundant samples using DBSCAN clustering on Mel-Frequency Cepstral Coefficients (MFCC) features, which are first flattened and then reduced in dimensionality using UMAP. In Step 2, we perform frame-level redundancy pruning for each utterance via utility pruning, which aims to eliminate irrelevant frames within each utterance. To the best of our knowledge, this is the first dataset pruning approach designed for Speech Classification tasks, demonstrating outstanding performance compared to classical general DP methods. Notably, for the Speech Emotion Recognition, our method achieves up to a 49.5% improvement in WA (Weighted Accuracy) on the MEAD dataset. For the Speaker Identification tasks, it results in a 41.9% reduction in EER (Equal Error Rate) on the VoxCeleb1 dataset.