We propose RCEM, a Robust Conversational search EMbedder that augments text embedders with LLM query reformulation capabilities while approximately preserving their original generalization performance. Unlike prior conversational dense retrieval approaches that learn direct conversation-to-passage matching, RCEM aligns conversations, prepended by special token, to LLM-rewritten queries, while attempting to preserve the original embedding space. The approximately preserved embedding space automatically maps the rewritten queries to the relevant passages. As a result, RCEM (1) reduces overfitting by aligning conversations to shorter rewritten queries rather than full passages, (2) eliminates the need for conversation-to-passage relevance labels for training, and (3) approximately preserves the original embedding space, enabling conversational retrieval over indexes built by the base embedder without re-indexing. Extensive experiments show that RCEM consistently outperforms prior approaches, improving up to 30% retrieval accuracy under distributional shift.