Vision-Language Models (VLMs) are increasingly applied to embodied task settings, yet it remains unclear whether their task-progress judgments are grounded in physical state comparison or in input-order shortcuts. We study a diagnostic two-frame setting: given two robot manipulation states and a task goal, the model must decide which state is closer to completion, or whether the two states are effectively equal. We observe a strong Forward-Backward Performance Gap: models often answer correctly in the natural forward order, but fail when the same two physical states are shown with their image slots swapped. This reversal is not intended to mimic the dominant deployment order of robot execution; rather, it is a counterfactual diagnostic for testing whether the selected physical state remains consistent under input-slot inversion. To reduce this order dependence, we introduce Equal-State supervision as a zero-point calibration signal, training models to output equal when no task-relevant progress difference is present instead of defaulting to a directional answer. From real manipulation videos, we construct the Embodied Causal-Temporal Reasoning (ECTR) dataset and propose Tri-Stage Weighted Alignment (TSWA), a staged fine-tuning procedure that strengthens spatial description, task-conditioned comparison, and answer calibration in sequence. On Qwen2.5-VL-7B, TSWA reduces GAP from 60.50% to 5.14% and raises overall accuracy from 44.66% to 80.19%. This comes with a small decrease in forward accuracy, but large gains in backward accuracy, equal-state recognition, and pairwise consistency, suggesting that TSWA replaces order-dependent correctness with more calibrated state comparison.