Recent LLM-based recommenders enhance language models with collaborative embeddings from user-item interactions, but making such embeddings available does not ensure their proper use during inference. Through a diagnostic attention analysis, we find that the utilization of collaborative embeddings is depth-dependent and alignment-sensitive, suggesting that LLMs need to balance their internal semantic knowledge with external collaborative knowledge. To address this issue, we propose SAILRec, an LLM-based recommender that improves this balance through dual-side semantic alignment and hierarchical attention steering. The former aligns item-side embeddings with item-text semantics and user-side embeddings with codebook-based semantic profiles, while the latter suppresses premature shallow-layer collaborative interference and strengthens collaborative evidence in deeper decision layers. Experiments on MovieLens-1M and Amazon-Book show that SAILRec consistently outperforms representative baselines, with ablation and masking analyses validating its key designs. Our implementation is available at https://github.com/XuPhoenixyear/SAILRec.

Sovereign AI deployments often require foundation models to operate under constraints overlooked by standard benchmarks: adherence to legislation, multilingual and multimodal grounding in local contexts, institution-specific safety behaviors, and on-premises deployment on power-constrained infrastructure. We present a production case study detailing the multi-stage engineering pipeline that spans trillion-token-scale continued pretraining, multi-billion-token alignment and post-training for a 123B-parameter natively multimodal foundation model, *Phoenix-VL 1.5 Medium*, optimized for Singapore public-sector use. On our novel localized evaluation suite and behavioral safety framework, the model substantially improves localized multimodal, statutory, and public-safety capability (a +0.118 average gain over the strongest baselines) while retaining broad general capability. We also share empirical analysis from a full production deployment currently serving over 30,000 users and processing ∼1 billion tokens over a two-month period. Ultimately, our findings demonstrate that sovereign AI is best approached as an end-to-end systems problem, where data curation, domain training, institutional alignment, and hardware layout interact and cannot be optimized in isolation.

The design choices in Transformer feed-forward neural networks have resulted in significant computational and parameter overhead. In this work, we emphasize the importance of hidden dimensions in designing lightweight FFNs, a factor often overlooked in previous architectures. Guided by this principle, we introduce PartialFormer, a parameter-efficient Transformer architecture utilizing multiple smaller FFNs to reduce parameters and computation while maintaining essential hidden dimensions. These smaller FFNs are integrated into a multi-head attention mechanism for effective collaboration. We also propose a tailored head scaling strategy to enhance PartialFormer’s capabilities. Furthermore, we present a residual-like attention calculation to improve depth scaling within PartialFormer. Extensive experiments on 9 translation tasks and 1 abstractive summarization task validate the effectiveness of our PartialFormer approach on machine translation and summarization tasks. Our code would be available at: https://github.com/zhengkid/PartialFormer.