Large language models (LLMs) increasingly serve as recommendation engines, yet existing preference control work primarily focuses on coarse-grained objectives such as safety, helpfulness, or writing style, while overlooking fine-grained provider preferences, i.e., user preferences over specific brands, platforms, or services. Such preferences are often multi-intent, context-dependent, and dynamically changing, making them difficult to control with prompting, fine-tuning, or activation steering methods. In this paper, we present COPILOT, a plug-and-play, training-free framework. It first decomposes each prompt into sub-tasks with corresponding user-specific provider preference configurations. Then it uses a diffusion language model to draft sub-task responses with hard preference constraints injected at the initial mask, which are finally completed into fluent answers by the target LLM. Experiments across code and text domains on three target LLMs show that COPILOT effectively improves provider preference satisfaction by up to 58.9% over the strongest baselines while reducing irrelevant provider mentions by up to 87.8% to ensure the recommendation quality. This work aims to lay a foundation for LLM recommendations that balance personalized experience with real-world application values.

Large Language Models (LLMs) have emerged as the new recommendation engines, surpassing traditional methods in both capability and scope, particularly in code generation. In this paper, we reveal a novel provider bias in LLMs: without explicit directives, these models show systematic preferences for services from specific providers in their recommendations (e.g., favoring Google Cloud over Microsoft Azure). To systematically investigate this bias, we develop an automated pipeline to construct the dataset, incorporating 6 distinct coding task categories and 30 real-world application scenarios. Leveraging this dataset, we conduct the first comprehensive empirical study of provider bias in LLM code generation across seven state-of-the-art LLMs, utilizing approximately 500 million tokens (equivalent to $5,000+ in computational costs). Our findings reveal that LLMs exhibit significant provider preferences, predominantly favoring services from Google and Amazon, and can autonomously modify input code to incorporate their preferred providers without users’ requests. Such a bias holds far-reaching implications for market dynamics and societal equilibrium, potentially contributing to digital monopolies. It may also deceive users and violate their expectations, leading to various consequences. We call on the academic community to recognize this emerging issue and develop effective evaluation and mitigation methods to uphold AI security and fairness.