Text embeddings incorporate additional rich semantic information; however, their performance in sequential recommendation still falls short of that of ID embeddings. Decomposing item embeddings along the spectrum of the collaborative item–item graph reveals a distinct discrepancy in their spectral energy distribution. Task-learned ID embeddings concentrate their energy heavily in the low- and mid-frequency directions of the collaborative graph topology, while textual representations spread more uniformly across the entire spectrum. Motivated by this insight, we propose CASG, a learnable polynomial graph filter that shifts text embeddings toward ID embeddings via collaboration-aware spectral guidance. Specifically, it leverages collaborative relationships to actively guide and filter the spectral energy distribution of frozen text embeddings, selectively preserving the frequency components aligned with the collaborative topology in a single offline pass. Across three datasets and two text encoders, CASG consistently surpasses learned ID embeddings, alignment-based, and fusion-based baselines at no online cost, suggesting that what semantic representations need is neither more alignment nor more fusion but collaboration-aware spectral guidance. Our source code is available at https://github.com/blueColdSea/CASG.