Arushi Goel
Author directory2026
Think, Look, and Revise: Inconsistency-Aware Visual Self-Correction in MLLMs
Yu Cheng | Arushi Goel | Hakan Bilen
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Yu Cheng | Arushi Goel | Hakan Bilen
Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing
Tool-augmented multimodal reasoning integrates external tools (e.g., object detection, depth estimation) into multimodal large language models (MLLMs) to address perceptual bottlenecks in complex visual tasks. However, existing approaches rarely verify tool outputs, limiting their ability to detect and recover from tool failures. We propose ReVISE, a framework that equips MLLMs with verification and dynamic error recovery for tool-augmented reasoning. ReVISE introduces (1) a curated training dataset that supervises reflective behaviors, enabling models to validate tool-derived evidence, reformulate queries when visual mismatches arise, and fall back to intrinsic grounding when external tools are unreliable; and (2) a reinforcement learning based targeted rewards that encourage internal reflection and penalize spatial misalignment. Experiments on several benchmarks demonstrate consistent improvements over existing methods, highlighting the importance of error detection and correction in tool-augmented multimodal reasoning.
Speech-Hands: A Self-Reflection Voice Agentic Approach to Speech Recognition and Audio Reasoning with Omni Perception
Zhen Wan | Chao-Han Huck Yang | Jinchuan Tian | Hanrong Ye | Ankita Pasad | Szu-Wei Fu | Arushi Goel | Ryo Hachiuma | Shizhe Diao | Kunal Dhawan | Sreyan Ghosh | Yusuke Hirota | Zhehuai Chen | Rafael Valle | Chenhui Chu | Shinji Watanabe | Boris Ginsburg | Yu-Chiang Frank Wang
Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
Zhen Wan | Chao-Han Huck Yang | Jinchuan Tian | Hanrong Ye | Ankita Pasad | Szu-Wei Fu | Arushi Goel | Ryo Hachiuma | Shizhe Diao | Kunal Dhawan | Sreyan Ghosh | Yusuke Hirota | Zhehuai Chen | Rafael Valle | Chenhui Chu | Shinji Watanabe | Boris Ginsburg | Yu-Chiang Frank Wang
Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
We introduce a voice-agentic framework that learns one critical omni-understanding skill: knowing when to trust itself versus when to consult external audio perception. Our work is motivated by a crucial yet counterintuitive finding: naively fine-tuning an omni-model on both speech recognition and external sound understanding tasks often degrades performance, as the model can be easily misled by noisy hypotheses. To address this, our framework, Speech-Hands, recasts the problem as an explicit self-reflection decision. This learnable reflection primitive proves effective in preventing the model from being derailed by flawed external candidates. We show that this agentic action mechanism generalizes naturally from speech recognition to complex, multiple-choice audio reasoning. Across the OpenASR leaderboard, which includes seven domain-diverse speech datasets, Speech-Hands consistently outperforms strong baselines by 12.1% WER on the OpenASR benchmark. The model also achieves 77.37% accuracy and high F1 on audio QA decisions, showing robust generalization and reliability across diverse audio question answering datasets. By unifying perception and decision-making, our work offers a practical path toward more reliable and resilient audio intelligence.
2023
Semi-supervised multimodal coreference resolution in image narrations
Arushi Goel | Basura Fernando | Frank Keller | Hakan Bilen
Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing
Arushi Goel | Basura Fernando | Frank Keller | Hakan Bilen
Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing
In this paper, we study multimodal coreference resolution, specifically where a longer descriptive text, i.e., a narration is paired with an image. This poses significant challenges due to fine-grained image-text alignment, inherent ambiguity present in narrative language, and unavailability of large annotated training sets. To tackle these challenges, we present a data efficient semi-supervised approach that utilizes image-narration pairs to resolve coreferences and narrative grounding in a multimodal context. Our approach incorporates losses for both labeled and unlabeled data within a cross-modal framework. Our evaluation shows that the proposed approach outperforms strong baselines both quantitatively and qualitatively, for the tasks of coreference resolution and narrative grounding.