Jungwuk Park

Author directory

2026

Mobile GUI agents often act confidently under ambiguous or incomplete user instructions-a common situation in real-world use. Even when explicitly offered a "call the user" action, open-weight agents ask for clarification in only 0.1% of cases on average, instead guessing the user’s intent and proceeding with actions. We name this failure Act-without-Asking. Existing step-level benchmarks largely miss this failure mode because they credit any ground-truth-matching action regardless of asking. We introduce MID-Bench, a step-level ambiguity benchmark constructed by rewriting clear instructions into ambiguous ones, with a strict clarification protocol: a user simulator answering only what is asked, and a scoring rule crediting a step only when the agent both asks an appropriate question and emits the correct final action. The Hard subset applies stricter masking to the action-critical detail in the instruction, making it harder to succeed without asking. Across four open-weight GUI agents, train-free cascades in which a gate decides whether to act or ask the user for clarification substantially improve ambiguous-task performance while preserving accuracy on clear instructions; LoRA fine-tuning on MID-Bench further combines effectively with these cascades.