Yu-Hsiang Liu

Author directory

2026

VLMs are increasingly positioned as daily assistants that perceive first-person environments, follow user dialogue, and decide how to help. Existing egocentric benchmarks mainly evaluate visual understanding in isolation, leaving open whether models can arbitrate between visual evidence and user-provided language when the two are helpful, irrelevant, or conflicting. We introduce EgoArgus, a human-annotated dataset for evaluating egocentric assistants on understanding and decision tasks in five dialogue-video daily scenarios. Our results demonstrate that it is still challenging for current VLMs as reliable egocentric assistants, which requires identifying which modality is trustworthy and deciding when intervention is warranted. Deeper analysis also shows that existing modality bias mitigation methods are quite restricted to enhance performance, providing insights to aid practioners into the deployment of current VLMs as daily assistants.
First-person assistance situations are difficult to collect with controlled timing and evidence: rare hazards should not be staged, while ordinary needs often depend on subtle visual and conversational context. We present VISTA, an interactive platform that treats scenario creation as an inspectable authoring workflow rather than a one-shot video prompt. A six-stage compiler turns a seed into typed intent, scene, event, interaction, rendering, and media artifacts; human reviewers can inspect revisions and provider requests before approving execution. We evaluate the rendered outputs, not the authoring interface. In a blinded study of 60 scenarios, 11 evaluators supplied 165 judgments. They selected the human-reviewed VISTA output as the best match to the seed in 52.1% of judgments, versus 22.4% for a one-pass seed-plus-assets baseline and 25.5% for seed-only generation. Case-balanced seed-match scores were 3.65, 3.27, and 3.22, respectively. These results support the complete workflow under the tested renderers; they do not establish interface usability, authoring efficiency, or a novel planning algorithm. The public artifact is a credential-free prepared replay with a narrated screencast.