Reliable multi-bit watermarking is essential for tracing the provenance of large language model (LLM) outputs. However, existing high-capacity schemes typically synchronize pseudo-random functions through discrete token n-gram matching, which makes them vulnerable to desynchronization under semantics-preserving paraphrasing attacks. To address this issue, we propose HSPMark, a hierarchical semantic projection marking mechanism for context-aware multi-bit watermarking, which shifts watermark synchronization from brittle token histories to continuous semantic anchors. HSPMark extracts stable contextual features using a pre-trained sentence-level encoder and a lightweight Robust Semantic MLP, then projects these features into context-conditioned pseudo-orthogonal bias matrices for multi-bit payload injection. To balance payload capacity and generation quality, HSPMark introduces an entropy-adaptive embedding mechanism that modulates logit bias according to predictive uncertainty. Extensive evaluations demonstrate that HSPMark improves robustness against paraphrase-induced desynchronization and local perturbations while maintaining a competitive trade-off among bit accuracy, payload capacity, and text quality compared with existing high-capacity baselines.

Multi-bit watermarking for large language models (LLMs) enables richer provenance information, but existing schemes remain unreliable for short texts because they rely on local or pre-specified allocation strategies. To address this limitation, we propose a Closed-Loop Error Feedback (CLEF) Multi-Bit Watermarking framework. By treating the LLM as a noisy channel, CLEF tracks the global feature state of generated tokens and dynamically measures its deviation from an ideal multidimensional target trajectory. The resulting dynamic error vector is then used to modulate the logits of candidate tokens via an inner-product scoring mechanism, biasing generation toward tokens that compensate for underrepresented payload dimensions. This closed-loop control enables robust multi-bit embedding with a simple, fully linear decoding process. Experiments on short-text generation show that CLEF improves bit extraction accuracy and complete message recovery rate over the evaluated multi-bit watermarking baselines. Furthermore, CLEF achieves higher bit extraction accuracy under different text perturbations while maintaining acceptable generation quality across downstream tasks such as text summarization and machine translation.