@inproceedings{wang-etal-2026-diagnosing-order,
title = "Diagnosing Order Dependence in Vision-Language Task-Progress Judgment with Equal-State Supervision",
author = "Wang, Can and
Yang, Xiaoda and
Zhang, Xueyi and
Tang, Menglan and
Xue, Jingyang and
Gao, Shenzhou and
Zhang, Jiahe and
Geng, Xiaoran and
Zhang, Jiangning and
Yuan, Hongyu and
Zhang, Peijian and
Hu, Xiaobin and
Lou, Jianwen",
editor = "Lee, Hung-Yi and
Dev, Sunipa and
Li, Jessy and
Elliott, Desmond",
booktitle = "Findings of the {A}ssociation for {C}omputational {L}inguistics: {EMNLP} 2026",
month = oct,
year = "2026",
address = "Budapest, Hungary",
publisher = "Association for Computational Linguistics",
url = "https://aclanthology.org/2026.findings-emnlp.792/",
pages = "16749--16773",
ISBN = "979-8-89176-462-0",
abstract = "Vision-Language Models (VLMs) are increasingly applied to embodied task settings, yet it remains unclear whether their task-progress judgments are grounded in physical state comparison or in input-order shortcuts. We study a diagnostic two-frame setting: given two robot manipulation states and a task goal, the model must decide which state is closer to completion, or whether the two states are effectively equal. We observe a strong Forward-Backward Performance Gap: models often answer correctly in the natural forward order, but fail when the same two physical states are shown with their image slots swapped. This reversal is not intended to mimic the dominant deployment order of robot execution; rather, it is a counterfactual diagnostic for testing whether the selected physical state remains consistent under input-slot inversion. To reduce this order dependence, we introduce Equal-State supervision as a zero-point calibration signal, training models to output equal when no task-relevant progress difference is present instead of defaulting to a directional answer. From real manipulation videos, we construct the Embodied Causal-Temporal Reasoning (ECTR) dataset and propose Tri-Stage Weighted Alignment (TSWA), a staged fine-tuning procedure that strengthens spatial description, task-conditioned comparison, and answer calibration in sequence. On Qwen2.5-VL-7B, TSWA reduces GAP from 60.50{\%} to 5.14{\%} and raises overall accuracy from 44.66{\%} to 80.19{\%}. This comes with a small decrease in forward accuracy, but large gains in backward accuracy, equal-state recognition, and pairwise consistency, suggesting that TSWA replaces order-dependent correctness with more calibrated state comparison."
}<?xml version="1.0" encoding="UTF-8"?>
<modsCollection xmlns="http://www.loc.gov/mods/v3">
<mods ID="wang-etal-2026-diagnosing-order">
<titleInfo>
<title>Diagnosing Order Dependence in Vision-Language Task-Progress Judgment with Equal-State Supervision</title>
</titleInfo>
<name type="personal">
<namePart type="given">Can</namePart>
<namePart type="family">Wang</namePart>
<role>
<roleTerm authority="marcrelator" type="text">author</roleTerm>
</role>
</name>
<name type="personal">
<namePart type="given">Xiaoda</namePart>
<namePart type="family">Yang</namePart>
<role>
<roleTerm authority="marcrelator" type="text">author</roleTerm>
</role>
</name>
<name type="personal">
<namePart type="given">Xueyi</namePart>
<namePart type="family">Zhang</namePart>
<role>
<roleTerm authority="marcrelator" type="text">author</roleTerm>
</role>
</name>
<name type="personal">
<namePart type="given">Menglan</namePart>
<namePart type="family">Tang</namePart>
<role>
<roleTerm authority="marcrelator" type="text">author</roleTerm>
</role>
</name>
<name type="personal">
<namePart type="given">Jingyang</namePart>
<namePart type="family">Xue</namePart>
<role>
<roleTerm authority="marcrelator" type="text">author</roleTerm>
</role>
</name>
<name type="personal">
<namePart type="given">Shenzhou</namePart>
<namePart type="family">Gao</namePart>
<role>
<roleTerm authority="marcrelator" type="text">author</roleTerm>
</role>
</name>
<name type="personal">
<namePart type="given">Jiahe</namePart>
<namePart type="family">Zhang</namePart>
<role>
<roleTerm authority="marcrelator" type="text">author</roleTerm>
</role>
</name>
<name type="personal">
<namePart type="given">Xiaoran</namePart>
<namePart type="family">Geng</namePart>
<role>
<roleTerm authority="marcrelator" type="text">author</roleTerm>
</role>
</name>
<name type="personal">
<namePart type="given">Jiangning</namePart>
<namePart type="family">Zhang</namePart>
<role>
<roleTerm authority="marcrelator" type="text">author</roleTerm>
</role>
</name>
<name type="personal">
<namePart type="given">Hongyu</namePart>
<namePart type="family">Yuan</namePart>
<role>
<roleTerm authority="marcrelator" type="text">author</roleTerm>
</role>
</name>
<name type="personal">
<namePart type="given">Peijian</namePart>
<namePart type="family">Zhang</namePart>
<role>
<roleTerm authority="marcrelator" type="text">author</roleTerm>
</role>
</name>
<name type="personal">
<namePart type="given">Xiaobin</namePart>
<namePart type="family">Hu</namePart>
<role>
<roleTerm authority="marcrelator" type="text">author</roleTerm>
</role>
</name>
<name type="personal">
<namePart type="given">Jianwen</namePart>
<namePart type="family">Lou</namePart>
<role>
<roleTerm authority="marcrelator" type="text">author</roleTerm>
</role>
</name>
<originInfo>
<dateIssued>2026-10</dateIssued>
</originInfo>
<typeOfResource>text</typeOfResource>
<relatedItem type="host">
<titleInfo>
<title>Findings of the Association for Computational Linguistics: EMNLP 2026</title>
</titleInfo>
<name type="personal">
<namePart type="given">Hung-Yi</namePart>
<namePart type="family">Lee</namePart>
<role>
<roleTerm authority="marcrelator" type="text">editor</roleTerm>
</role>
</name>
<name type="personal">
<namePart type="given">Sunipa</namePart>
<namePart type="family">Dev</namePart>
<role>
<roleTerm authority="marcrelator" type="text">editor</roleTerm>
</role>
</name>
<name type="personal">
<namePart type="given">Jessy</namePart>
<namePart type="family">Li</namePart>
<role>
<roleTerm authority="marcrelator" type="text">editor</roleTerm>
</role>
</name>
<name type="personal">
<namePart type="given">Desmond</namePart>
<namePart type="family">Elliott</namePart>
<role>
<roleTerm authority="marcrelator" type="text">editor</roleTerm>
</role>
</name>
<originInfo>
<publisher>Association for Computational Linguistics</publisher>
<place>
<placeTerm type="text">Budapest, Hungary</placeTerm>
</place>
</originInfo>
<genre authority="marcgt">conference publication</genre>
<identifier type="isbn">979-8-89176-462-0</identifier>
</relatedItem>
<abstract>Vision-Language Models (VLMs) are increasingly applied to embodied task settings, yet it remains unclear whether their task-progress judgments are grounded in physical state comparison or in input-order shortcuts. We study a diagnostic two-frame setting: given two robot manipulation states and a task goal, the model must decide which state is closer to completion, or whether the two states are effectively equal. We observe a strong Forward-Backward Performance Gap: models often answer correctly in the natural forward order, but fail when the same two physical states are shown with their image slots swapped. This reversal is not intended to mimic the dominant deployment order of robot execution; rather, it is a counterfactual diagnostic for testing whether the selected physical state remains consistent under input-slot inversion. To reduce this order dependence, we introduce Equal-State supervision as a zero-point calibration signal, training models to output equal when no task-relevant progress difference is present instead of defaulting to a directional answer. From real manipulation videos, we construct the Embodied Causal-Temporal Reasoning (ECTR) dataset and propose Tri-Stage Weighted Alignment (TSWA), a staged fine-tuning procedure that strengthens spatial description, task-conditioned comparison, and answer calibration in sequence. On Qwen2.5-VL-7B, TSWA reduces GAP from 60.50% to 5.14% and raises overall accuracy from 44.66% to 80.19%. This comes with a small decrease in forward accuracy, but large gains in backward accuracy, equal-state recognition, and pairwise consistency, suggesting that TSWA replaces order-dependent correctness with more calibrated state comparison.</abstract>
<identifier type="citekey">wang-etal-2026-diagnosing-order</identifier>
<location>
<url>https://aclanthology.org/2026.findings-emnlp.792/</url>
</location>
<part>
<date>2026-10</date>
<extent unit="page">
<start>16749</start>
<end>16773</end>
</extent>
</part>
</mods>
</modsCollection>
%0 Conference Proceedings
%T Diagnosing Order Dependence in Vision-Language Task-Progress Judgment with Equal-State Supervision
%A Wang, Can
%A Yang, Xiaoda
%A Zhang, Xueyi
%A Tang, Menglan
%A Xue, Jingyang
%A Gao, Shenzhou
%A Zhang, Jiahe
%A Geng, Xiaoran
%A Zhang, Jiangning
%A Yuan, Hongyu
%A Zhang, Peijian
%A Hu, Xiaobin
%A Lou, Jianwen
%Y Lee, Hung-Yi
%Y Dev, Sunipa
%Y Li, Jessy
%Y Elliott, Desmond
%S Findings of the Association for Computational Linguistics: EMNLP 2026
%D 2026
%8 October
%I Association for Computational Linguistics
%C Budapest, Hungary
%@ 979-8-89176-462-0
%F wang-etal-2026-diagnosing-order
%X Vision-Language Models (VLMs) are increasingly applied to embodied task settings, yet it remains unclear whether their task-progress judgments are grounded in physical state comparison or in input-order shortcuts. We study a diagnostic two-frame setting: given two robot manipulation states and a task goal, the model must decide which state is closer to completion, or whether the two states are effectively equal. We observe a strong Forward-Backward Performance Gap: models often answer correctly in the natural forward order, but fail when the same two physical states are shown with their image slots swapped. This reversal is not intended to mimic the dominant deployment order of robot execution; rather, it is a counterfactual diagnostic for testing whether the selected physical state remains consistent under input-slot inversion. To reduce this order dependence, we introduce Equal-State supervision as a zero-point calibration signal, training models to output equal when no task-relevant progress difference is present instead of defaulting to a directional answer. From real manipulation videos, we construct the Embodied Causal-Temporal Reasoning (ECTR) dataset and propose Tri-Stage Weighted Alignment (TSWA), a staged fine-tuning procedure that strengthens spatial description, task-conditioned comparison, and answer calibration in sequence. On Qwen2.5-VL-7B, TSWA reduces GAP from 60.50% to 5.14% and raises overall accuracy from 44.66% to 80.19%. This comes with a small decrease in forward accuracy, but large gains in backward accuracy, equal-state recognition, and pairwise consistency, suggesting that TSWA replaces order-dependent correctness with more calibrated state comparison.
%U https://aclanthology.org/2026.findings-emnlp.792/
%P 16749-16773
Markdown (Informal)
[Diagnosing Order Dependence in Vision-Language Task-Progress Judgment with Equal-State Supervision](https://aclanthology.org/2026.findings-emnlp.792/) (Wang et al., Findings 2026)
ACL
- Can Wang, Xiaoda Yang, Xueyi Zhang, Menglan Tang, Jingyang Xue, Shenzhou Gao, Jiahe Zhang, Xiaoran Geng, Jiangning Zhang, Hongyu Yuan, Peijian Zhang, Xiaobin Hu, and Jianwen Lou. 2026. Diagnosing Order Dependence in Vision-Language Task-Progress Judgment with Equal-State Supervision. In Findings of the Association for Computational Linguistics: EMNLP 2026, pages 16749–16773, Budapest, Hungary. Association for Computational Linguistics.