In existing memory benchmarks for Large Language Models (LLMs), the evaluated dialogue sessions often lack long-term semantic consistency, and the underlying personas tend to be flat and static. Furthermore, in real-world scenarios, interactions between users and assistants involve more diverse, heterogeneous data streams, such as documents and emails. These shortcomings significantly limit the realism and effectiveness of current evaluations. To address these limitations, we introduce RHELM ( ̲Realistic,  ̲Heterogeneous, and  ̲Evolving  ̲Long-term  ̲Memory). Driven by meticulously crafted user profiles and a novel LOOP (p ̲Lan-r ̲Ollout-ev ̲Olve- ̲Prune) module, we construct realistic dialogues across diverse interaction scenarios that exhibit dynamic temporal evolution and long-term coherence. Crucially, these dialogues are deeply integrated with heterogeneous external sources synchronized with the user’s temporal event trajectory. The resulting benchmark encompasses challenging question-answer pairs spanning seven inquiry types, with each question mapping to at least one of 27 critical memory characteristics that we identify as essential yet underexplored in current research. Comprehensive experiments across full-context models, retrieval-augmented generation (RAG) methods, and representative memory frameworks reveal that contemporary approaches still expose critical weaknesses in complex, real-world settings, particularly in resolving multi-source aggregation and real-world contextual reasoning.

Tables are widespread in real-world scenarios. In particular, hierarchical tables encode much richer structural semantics; however, their complex layouts pose severe challenges to Large Language Models (LLMs). The core difficulty lies in precisely aligning complex topological structures with multi-level semantic dependencies, where alignment failures often lead to cross-level aggregation errors and hallucinations. To address this challenge, we propose STAR-Table, a framework that synergizes visual perception with symbolic reasoning. Unlike traditional text-only approaches, STAR-Table leverages OCR models to extract pixel-level spatial coordinates, thereby establishing explicit Spatial-Textual AnchoRs. These anchored spatial priors are formulated as explicit structural hints to guide LLMs in reconstructing the hierarchical topology of tables. Based on the reconstructed topology, complex visual layouts are decomposed into one or more distinct sub-tables, each of which is further mapped into a deterministic flattened database. Finally, we design an adaptive QA module that fuses Vision-Language Model (VLM) semantic summaries with multi-step SQL execution to support robust table-based reasoning. Experimental results across multiple benchmarks demonstrate that STAR-Table achieves superior performance, significantly outperforming specialized fine-tuned models and advanced in-context learning paradigms.

Data curation for large language models (LLMs) has been extensively studied for both the pre-training and post-training stages, yet the intermediate mid-training phase—bridging general pretraining and task-specific fine-tuning—remains critically underexplored. Most existing mid-training data for state-of-the-art models remains proprietary. Meanwhile, the Hugging Face platform hosts a huge amount of community-contributed datasets, yet no systematic effort has been made to refine this rich ecosystem into a unified high-quality corpus. We present HuggingBrew, a large-scale mid-training corpus constructed through a principled, multi-step curation pipeline applied to the massive publicly accessible Hugging Face datasets. The pipeline operates at two complementary granularities: dataset-level operations act on each dataset independently, while corpus-level operations are applied jointly across all retained corpus. We construct three progressively curated variants—\textsf{Draught}, \textsf{Craft}, and \textsf{Reserve}—containing 1.6T, 450B, and 40B tokens, respectively. Continual pretraining experiments under a 100B-token budget show that series yields substantial improvements over baselines across mathematical reasoning, general knowledge, and code generation, validating the untapped value of community-contributed data for mid-training.

Existing news recommendation methods usually learn news representations solely based on news titles. To sufficiently utilize other fields of news information such as category and entities, some methods treat each field as an additional feature and combine different feature vectors with attentive pooling. With the adoption of large pre-trained models like BERT in news recommendation, the above way to incorporate multi-field information may encounter challenges: the shallow feature encoding to compress the category and entity information is not compatible with the deep BERT encoding. In this paper, we propose a multi-task method to incorporate the multi-field information into BERT, which improves its news encoding capability. Besides, we modify the gradients of auxiliary tasks based on their gradient conflicts with the main task, which further boosts the model performance. Extensive experiments on the MIND news recommendation benchmark show the effectiveness of our approach.