Gloss-free sign language translation (SLT) aims to translate sign videos directly into spoken-language sentences without relying on costly gloss annotations. However, when supervision is available only at the sentence level, the correspondence between visual dynamics and linguistic content remains weak. We introduce TransAlign-SLT, a lightweight fully gloss-free framework that derives complementary temporal and semantic supervision from the available translations. Auxiliary TranslationCTC provides coarse temporal regularization, while Masked Entropic Optimal Transport (MOT) aligns visual temporal features with contextualized word-level representations from a frozen, corpus-adapted BERT encoder. Since the semantic branch is used only during training, the inference pipeline remains lightweight and autoregressive. TransAlign-SLT achieves 23.54 BLEU-4 / 48.46 ROUGE-L on PHOENIX14T and 16.39 BLEU-4 / 44.68 ROUGE-L on CSL-Daily, with 45.6M inference parameters and 123 GFLOPs. Ablation studies show that temporal modeling, semantic alignment, and decayed TranslationCTC supervision provide complementary benefits. Overall, the results show that useful visual-language structure can be learned directly from translation supervision without glosses, pseudo-glosses, or a large pretrained language decoder at inference.