Code-switching is a pervasive linguistic phenomenon in global communication, yet modern information retrieval systems remain predominantly designed for, and evaluated within, monolingual contexts. To bridge this critical disconnect, we present a holistic study dedicated to code-switching IR. We introduce CSR-L (Code-Switching Retrieval benchmark-Lite), constructing a dataset via human annotation to capture the authentic naturalness of mixed-language queries. Our evaluation across statistical, dense, and late-interaction paradigms reveals that code-switching acts as a fundamental performance bottleneck, degrading the effectiveness of even robust multilingual models. We demonstrate that this failure stems from substantial divergence in the embedding space between pure and code-switched text. Scaling this investigation, we propose CS-MTEB, a comprehensive benchmark covering 11 diverse tasks, where we observe performance declines of up to 27%. Finally, we show that standard multilingual techniques like vocabulary expansion are insufficient to resolve these deficits completely. These findings underscore the fragility of current systems and establish code-switching as a crucial frontier for future IR optimization.

Instruction-following information retrieval (IF-IR) requires retrievers to satisfy both topical intent and instance-specific relevance constraints. We propose dual-view training, a data synthesis strategy based on polarity reversal. Given a query, an instruction-relevant document, and a topically related document excluded by that instruction, an LLM generates a complementary instruction under which the two documents exchange relevance labels. The resulting paired views directly penalize instruction-agnostic representations. At a matched 880k-example budget, replacing the non-instruction half of a Promptriever-style joint-data recipe with polarity-reversed examples improves FollowIR p-MRR from 5.27 to 8.30 (+57.5%) on a 305M-parameter encoder while increasing its instruction-conditioned Score from 20.85 to 21.38. The relative p-MRR gain repeats on a 560M-parameter backbone. The proposed recipe also improves InfoSearch nDCG@5 and shows no average degradation on four evaluated BEIR datasets with plain queries. A size-matched paraphrase control does not reproduce the instruction-sensitivity gains, supporting the conclusion that label reversal, rather than additional LLM-generated wording, provides the key supervision signal.

Neural ranking models have become core components of modern information retrieval systems and important building blocks of AI systems such as retrieval-augmented generation (RAG) pipelines. However, their robustness remains insufficiently understood in the presence of large language models (LLMs), which can generate fluent and deceptive content at scale. This work investigates the vulnerability of neural ranking models to corpus poisoning attacks, in which an adversary injects a small number of maliciously crafted documents into the corpus to distort ranking behavior. We propose VerTox, the first framework to formulate corpus poisoning as a verifiable reward-guided reinforcement learning (RLVR) problem. By explicitly coupling ranking distortion with factual corruption through specialized reward shaping, we fine-tune compact LLMs into adversarial generators. Experiments demonstrate that our method achieves near-perfect attack success rates, producing adversarial documents that frequently rank higher than target documents across major neural ranking architectures, as well as a proprietary commercial embedding model. The generated adversarial documents are fluent and exhibit low perplexity, making them difficult to detect. Furthermore, by explicitly encouraging factual corruption, our adversarial documents significantly degrade the performance of a downstream RAG application.

Text embedding models are central to information retrieval and deep research tasks, and their behaviors are strongly shaped by the training objective. While recent systems have largely converged on InfoNCE-style loss functions, there have been continued attempts to design additional loss functions beyond this core objective. One such component is spread-out loss, a batch-level penalty on high pairwise similarity that encourages embeddings to occupy the representation space more evenly. However, in existing systems, this design is usually combined with many other design choices and has not been sufficiently ablated. To isolate its effect, we use spread-out loss as a controlled lens for studying geometric regularization in text embedding training. Across 13 retrieval benchmarks, spread-out improves fine-tuned retrievers over contrastive baselines across architectures and model sizes. We then diagnose where these gains arise and find that they concentrate on short queries, where spread-out reduces query-side directional crowding. Based on this diagnosis, we propose QUARTO, a Query-length Adaptive spread-out loss that uses batch composition to scale the full regularizer more strongly for short-query-dominated batches and less strongly for longer-query-dominated batches. QUARTO further improves retrieval performance over fixed-weight spread-out. Overall, our findings point to the potential of embedding-space regularization and suggest that text-embedding training should explore a broader design space of objectives beyond InfoNCE-style contrastive learning. Project page: https://github.com/JoeZhou-encoding/Spreadout_QUARTO

In search settings, calibrating the scores during the ranking process to quantities such as click-through rates or relevance levels enhances a system’s usefulness and trustworthiness for downstream users. While previous research has improved this notion of calibration for low complexity learning-to-rank models, the larger data demands and parameter count specific to modern neural text rankers produce unique obstacles that hamper the efficacy of methods intended for the learning-to-rank setting.This paper proposes exploiting large language models (LLMs) to provide relevance and uncertainty signals for these neural text rankers to produce scale-calibrated scores through Monte Carlo sampling of natural language explanations (NLEs). Our approach transforms the neural ranking task from ranking textual query-document pairs to ranking corresponding synthesized NLEs. Comprehensive experiments on two popular document ranking datasets show that the NLE-based calibration approach consistently outperforms past calibration methods and LLM-based methods for ranking, calibration, and query performance prediction tasks.

Multilingual models aim for language-invariant representations but still prominently encode language identity. This, along with the scarcity of high-quality parallel retrieval data, limits their performance in retrieval. We introduce LANCER, a multi-task learning framework that improves language-invariant dense retrieval by reducing language-specific signals in the embedding space. Leveraging the notion of linear concept erasure, we design a loss function that penalizes cross-correlation between representations and their language labels. LANCER leverages only English retrieval data and general multilingual corpora, training models to focus on language-invariant retrieval by semantic similarity without necessitating a vast parallel corpus. Experimental results on various datasets show our method consistently improves over baselines, with extensive analyses demonstrating greater language agnosticism.