Temporal Knowledge Graph (TKG) extrapolation aims to predict future facts from observed historical events. Existing methods usually achieve this goal by jointly modeling local historical evolution and long-range historical contexts. First, local evolution modeling often summarizes local history into an overall evolving representation, where query-relevant local deviations may not be sufficiently exploited as useful predictive features. Second, existing long-range history modeling often treats historical recurrence as a stable cue, while overlooking that its reliability may vary across relations and recent contexts. To address these issues, we propose DyHist, a query-aware historical dynamics modeling framework for TKG extrapolation. DyHist captures local dynamics by separating recent evidence from medium-range context and explicitly modeling their query-relevant deviations as predictive features. It further uses a relation-aware historical prior to calibrate candidate scores with time-decayed support, recent trends, and relation-specific strength, enabling recurrence signals to contribute differently across relations and recent contexts. Experiments on four benchmarks show that DyHist achieves the best MRR against recent baselines. Code is available at https://github.com/PANPANKK/DyHist.

Vision-Language Pretraining (VLP) for 3D CT scans is severely constrained by the massive GPU memory required for high-resolution volumes and large contrastive batches. We propose ECT-CLIP, a resource-efficient CT VLP framework that achieves negative scaling—scaling up both the quantity and quality of negative samples—under limited computational resources. To scale the quantity, we exploit the memory asymmetry between CT and text encoding to decouple the image and text batch sizes, thereby enlarging the negative pool. To scale the quality, we design two complementary strategies: (1) deterministically separating false negatives from informative hard negatives via UMLS concept normalization, and (2) fusing multiple frozen medical text encoders to enrich semantic diversity without extra training cost. Trained on a single consumer-grade RTX 4090 GPU, ECT-CLIP achieves state-of-the-art zero-shot classification AUCs of 83.4% on CT-RATE and 75.0% on the external RAD-ChestCT benchmark, outperforming prior methods that require 4–8 high-performance GPUs. Extensive ablation studies validate the effectiveness of each negative-scaling component.

Multimodal learning serves as a promising approach for applications with diverse information sources. However, there are numerous challenges when scaling up multimodal learning data from all modalities due to availability or varied cost of data collection. We formulate and study multimodal learning under limited data collection budgets as a modality selection problem, demonstrating that selectively collecting a subset of modalities and synthesizing the missing ones can match or outperform models trained with uniformly augmented full modalities. Our research problem is formulated as: given a limited data collection budget, how to find the appropriate modalities to collect new data and generate for the rest to maximize model performance gain? To answer this, we propose a new paradigm - Select the Key modality, then generate the rest to enable learning with limited data (SK-ll). SK-ll contains two essential components: (1) Select the key. We propose a modality importance indicator to find the optimal modalities by assessing their single modal marginal contribution and cross-modal interactions. (2) Generate the rest. We substitute with generated embeddings for the rest of modalities. We conducted extensive experiments applying SK-ll across affective computing, healthcare with diverse multimodal learning backbones, obtaining average accuracy gains of 2.37%, 6.55%, 6.73% on MOSI, MOSEI, ADNI respectively. Meanwhile, we present interesting empirical insights such as the data efficiency. Codes are provided in https://github.com/UNITES-Lab/SK-ll.

We introduce LM-Lexicon, an innovative definition modeling approach that incorporates data clustering, semantic expert learning, and model merging using a sparse mixture-of-experts architecture. By decomposing the definition modeling task into specialized semantic domains, where small language models are trained as domain experts, LM-Lexicon achieves substantial improvements (+7% BLEU score compared with the prior state-of-the-art model) over existing methods on five widely used benchmarks. Empirically, we demonstrate that 1) the clustering strategy enables fine-grained expert specialization with nearly 10% improvement in definition quality; 2) the semantic-aware domain-level routing mechanism achieves higher expert efficacy (+1%) than conventional token-level routing; and 3) further performance gains can be obtained through test-time compute and semantic expert scaling. Our work advances definition modeling while providing insights into the development of efficient language models for semantic-intensive applications.

Large Language Models (LLMs) often suffer from “Reasoning Collapse” on challenging mathematical reasoning tasks, where stochastic sampling produces lexical variations of the same erroneous logic rather than genuine semantic exploration. We observe that failed reasoning traces are often associated with a low-rank bias manifold in the model’s hidden-state geometry, which reduces exploration toward corrective solution directions. To address this, we propose Spectral Orthogonal Exploration (SOE), a geometric inference framework under a “Student Guides Teacher” paradigm. Instead of using a weak auxiliary agent for imitation, SOE uses it as an orthogonal probe to introduce semantically heterogeneous reasoning signals into the teacher’s orthogonal complement of its dominant subspace. This intervention steers the teacher toward more diverse reasoning trajectories and improves exploration beyond standard sampling. Experiments on mathematical benchmarks show that SOE improves average accuracy by 62.4% and average sampling efficiency by 113.7% over baseline methods, suggesting that geometric interventions can be effective for mitigating reasoning collapse in mathematical reasoning. We further provide preliminary evidence that SOE is also effective on logic and code generation benchmarks. Code is available at https://github.com/dayuwang401/spectral-orthogonal-exploration.