The prevalence of rapidly evolving slang, neologisms, and highly stylized expressions in informal user-generated text, particularly on Chinese social media, poses significant challenges for Machine Translation (MT) benchmarking. Specifically, we identify two primary obstacles: (1) data scarcity, as high-quality parallel data requires bilingual annotators familiar with platform-specific slang, and stylistic cues; and (2) metric limitations, where traditional evaluators like COMET often fail to capture stylistic fidelity and non-standard expressions. To bridge these gaps, we first introduce CSM-MTBench, an expert-curated benchmark covering five Chinese→foreign language directions and comprising 10,915 total instances, organized into two subsets: **Fun Posts**, featuring context-rich content with frequent slang and neologisms, and **Social Snippets**, focusing on concise, emotion- and style-driven expressions. Next, we propose tailored evaluation approaches for each subset, including slang translation success rate and style-preservation metrics combining embeddings and LLM-asa-judge evaluation. Experiments on over 20 models reveal substantial variation in how current MT systems handle semantic fidelity, slang translation, and stylistic nuances. We further look into model behaviors through prompt sensitivity analysis and qualitative case studies, highlighting the challenges of translating these informal expressions. CSM-MTBench thus serves as a rigorous testbed for advancing MT systems capable of mastering real-world Chinese social media texts.

Data mixture selection is critical for Large Language Model pretraining. Existing methods such as RegMix select a single static mixture by fitting a regression model on small-scale proxy runs. We propose RegMix-D, a simple extension of RegMix to dynamic mixing. Our key observation is that proxy runs produce not only endpoint losses, but also full loss trajectories, which can be used to further improve data mixture. By training regression model on these trajectories, we can predict optimal mixtures at multiple training stages. RegMix-D supports two deployment modes: an *offline* variant that generates a complete mixture schedule before target training, and an *online* variant that adapts the mixture during training using observed loss. Experiments on 25B tokens of the Pile dataset with a 1B parameter target model show that RegMix-D consistently improves over baselines across 13 downstream tasks, and the improvement carries over to a 7B target model on 100B tokens. RegMix-D also remains proxy-efficient: it surpasses RegMix even with only 128 proxy models (25% of RegMix’s proxy compute budget).

Multimodal sentence embedding models typically leverage image-caption pairs in addition to textual data during training. However, such pairs often contain noise, including redundant or irrelevant information on either the image or caption side. To mitigate this issue, we propose MCSEO, a method that enhances multimodal sentence embeddings by incorporating fine-grained object-phrase alignment alongside traditional image-caption alignment. Specifically, MCSEO utilizes existing detection and segmentation models to extract accurate object-phrase pairs, which are then used to optimize a proposed contrastive learning objective tailored to object-phrase correspondence. Experimental results on semantic textual similarity (STS) tasks across different backbone models demonstrate that MCSEO outperforms image-caption-only baselines on average, and that it is complementary to existing methods: combining MCSEO with KDMCSE yields further gains. A random-pairing control and a manual evaluation of the extracted pairs additionally indicate that the improvements stem from correct object-phrase alignment.

Diffusion Large Language Models (DLLMs) have recently emerged as a parallel alternative to autoregressive models, offering bidirectional context modeling and parallel decoding. However, developing highly performant DLLMs is hindered by lifecycle challenges, including suboptimal timestep weighting in pre-training, train-inference distribution gaps in post-training, and underutilized caching during inference. To address these obstacles, we introduce ReDLLM, a 30B-parameter Mixture-of-Experts DLLM. It incorporates a time-aware weighting strategy during continual pre-training that scales gradients according to denoising difficulty. Furthermore, we propose two block-oriented refinements: a terminal-block treatment to narrow the train-inference gap, and a block-aware inference pipeline featuring inline block prefill and block-level scheduling to maximize KV-cache reuse and parallelization. Extensive evaluations demonstrate that ReDLLM surpasses existing DLLMs at the same active-parameter scale, achieves generation quality competitive with 100B DLLMs, and attains the highest inference throughput among evaluated models.

Neologism-aware machine translation aims to translate source sentences containing neologisms into target languages.This field remains underexplored compared with general machine translation (MT).In this paper, we propose an agentic framework, NeoAMT, for neologism-aware machine translation equipped with a Wiktionary-based search toolkit.Specifically, we first construct a dedicated dataset for neologism-aware machine translation and build a search toolkit grounded in Wiktionary.The dataset covers 16 languages and 75 translation directions in total, derived from approximately 10 million records of an English Wiktionary dump.The retrieval corpus of the search toolkit is also constructed from around 3 million cleaned records of the same dump.We then leverage the dataset and toolkit to train a translation agent via reinforcement learning (RL) and to evaluate the accuracy of neologism-aware machine translation.Furthermore, we propose an RL training framework featuring a novel reward design and an adaptive rollout generation strategy that exploits translation difficulty to further improve the translation quality of translation agents using our search toolkit.

Recently, many works have been attempting to adapt Large Language Models (LLMs) for sentence embedding, with most of them fine-tuning LLMs towards the contrastive objective and enabling bi-directional attention for better performance, using LoRA to address the large model scale.In this work, we suggest that this adaptation can also be simply and effectively achieved using causal attention and with even fewer trainable parameters through soft prompt tuning, as an alternative to fine-tuning with LoRA and other methods with extra post-training tasks.Our method only optimizes a few learnable tokens while keeping the rest of the model frozen.Through experiments on a diverse set of evaluation tasks, we find that simply tuning only a few tokens can achieve a competitive performance with that of fine-tuning with LoRA. The percentage of trainable parameters can be reduced to less than 0.001%. Moreover, we also demonstrate that turning causal attention to bi-directional attention with or without extra post-training tasks does not provide additional benefit when soft prompt tuning is applied, suggesting that causal attention can be naturally used in decoder-only LLMs for sentence embedding adaptation.

The field of cross-lingual sentence embeddings has recently experienced significant advancements, but research concerning low-resource languages has lagged due to the scarcity of parallel corpora. This paper shows that cross-lingual word representation in low-resource languages is notably under-aligned with that in high-resource languages in current models. To address this, we introduce a novel framework that explicitly aligns words between English and eight low-resource languages, utilizing off-the-shelf word alignment models. This framework incorporates three primary training objectives: aligned word prediction and word translation ranking, along with the widely used translation ranking. We evaluate our approach through experiments on the bitext retrieval task, which demonstrate substantial improvements on sentence embeddings in low-resource languages. In addition, the competitive performance of the proposed model across a broader range of tasks in high-resource languages underscores its practicality.

Learning multilingual sentence embeddings is a fundamental task in natural language processing. Recent trends in learning both monolingual and multilingual sentence embeddings are mainly based on contrastive learning (CL) among an anchor, one positive, and multiple negative instances. In this work, we argue that leveraging multiple positives should be considered for multilingual sentence embeddings because (1) positives in a diverse set of languages can benefit cross-lingual learning, and (2) transitive similarity across multiple positives can provide reliable structural information for learning.In order to investigate the impact of multiple positives in CL, we propose a novel approach, named MPCL, to effectively utilize multiple positive instances to improve the learning of multilingual sentence embeddings. Experimental results on various backbone models and downstream tasks demonstrate that MPCL leads to better retrieval, semantic similarity, and classification performance compared to conventional CL. We also observe that in unseen languages, sentence embedding models trained on multiple positives show better cross-lingual transfer performance than models trained on a single positive instance.